Digital video file generation

ABSTRACT

The present disclosure involves systems, software, and computer implemented methods for personalizing media content. Personalization digital media templates can be retrieved that designate (i) particular types of media content to be combined with particular preselected digital video files and (ii) particular times within particular preselected digital video files at which the particular types of media content are to be combined with the particular preselected digital video files, the particular times being relative to the times within the preselected digital video files at which emotionally impactful moments occur. Digital media content can be assembled for the personalized digital video files using particular preselected digital video files, the digital media templates, and the personal digital media file. The assembled digital media content can be encoded using a common video codec to generate the personalized digital video files. The personalized digital video files can be stored and transmitted to a client computing device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/147,048, filed Sep. 28, 2018, which is a continuation of U.S.application Ser. No. 15/860,568, filed Jan. 2, 2018 (now U.S. Pat. No.10,110,942), which claims the benefit of priority to U.S. ProvisionalApplication Ser. No. 62/441,081, filed Dec. 30, 2016, entitled MEDIAPERSONALIZATION AND SOCIAL MEDIA PLATFORM, the entire contents of whichare hereby incorporated by reference. This application includes a commondisclosure and priority claim with U.S. application Ser. No. 16/146,484,filed Feb. 7, 2019, which claims priority to U.S. application Ser. No.15/860,545, filed Jan. 2, 2018 (now U.S. Pat. No. 10,123,065), whichclaims the benefit of priority to U.S. Provisional Application Ser. No.62/441,081, filed Dec. 30, 2016, entitled MEDIA PERSONALIZATION ANDSOCIAL MEDIA PLATFORM, the entire contents of which are herebyincorporated by reference.

TECHNICAL FIELD

This specification relates to technology for efficiently generatingdigital video files to include personalized content.

BACKGROUND

Media personalization software, such as video and audio editingsoftware, provides users with features that can be used to combine mediacontent (e.g., videos, audio, images, text) in various ways. Forexample, video editing software can allow a user to trim video clips,combine video clips, add audio tracks, and to add graphics, images, andtext. Video editing software can rely on users to retrieve and identifyvideo clips for editing, the manner and timing with which video clipsare combined, and the ultimate composition of the final video.

While social media platforms (e.g., FACEBOOK, TWITTER, LINKEDIN,INSTAGRAM) vary in their approach to online interactions between users,social media platforms generally provide features through which userscan share information and interact with a broader collection of users onthe platform. For example, users on social media platforms can postcontent that is then distributed to other users on the social mediaplatform, such as friends, followers, or fans of the user posting thecontent. Such distribution of content among users can be non-private inthat it is broadcast among a broad group of users, which can sometimesbe people without any sort of social connection to the posting user.

SUMMARY

This document generally describes improved technology for personalizingmedia content to more consistently and efficiently generate emotionallyimpactful personalized media content. Computer systems, techniques, anddevices are described for automating the personalization of mediacontent to ensure that personalized content is presented at appropriatetimes and places on the underlying media content that is beingpersonalized, and to ensure that the quality of the underlying mediacontent is undisturbed by the personalization. For example, music andvideos often have “chill” moments that are emotionally impactful forlisteners/viewers, such as the chorus in the song “Let It Go” from themovie Frozen. The technology described in this document can automategeneration of a personalized “mediagram” (personalized media contentconveying a message) based on an excerpt of the song “Let It Go” withpersonalization (e.g., text, images, video, audio) at appropriate timesand locations around the song's chorus to provide an emotionallyimpactful message that leverages the chill moment from the song.

This document also generally describes an improved social platform toenhance the quality of social interactions and relationships amongusers. Such a social platform can include a variety of features, such asprivate communication channels between users centered around the userrelationships, relationship concierge features to facilitate and improvethe quality of social interactions, time delays between socialinteractions to alleviate the pressure and stress on users of needing torespond quickly, temporary social interactions that are inaccessible tousers involved in the interactions after a threshold period of time,private group communication channels and group relationship conciergefeatures, relationship scoring features, personalized media contentcreation and distribution features, interactive and social emotionalwell-being meters through which users can identify their own emotionalstate and the states of other users, and/or combinations thereof. Suchfeatures can assist users in building and maintaining strongrelationships with other users.

In one implementation, a method for automatically generatingpersonalized videos includes outputting, in a user interface on a clientcomputing device, information identifying a plurality of preselectedvideos, wherein each of the plurality of preselected videos (i) areexcerpts of longer videos and (ii) include at least one emotionallyimpactful moment; receiving, through an input subsystem on the clientcomputing device, selection of a particular video from the plurality ofpreselected videos; retrieving, by the client computing device and inresponse to receiving the selection, the particular video and apersonalization template for the particular video, wherein thepersonalization template designates particular types of media content tobe combined with the video at particular locations to maximize the atleast one emotionally impactful moment for an intended recipient;outputting, by the client computing device, a plurality of input fieldsprompting the user to select the particular types of media content fromone or more personal repositories of media content; automaticallyretrieving, in response to user selections through the plurality ofinput fields, personal media content from the one or more personalrepositories of media content; automatically assembling, by the clientcomputing device and without user involvement beyond the userselections, a personalized video that includes the particular video andthe personal media content at the particular locations in the video; andoutputting, by the client computing device, the personalized video.

Such an implementation can optionally include one or more of thefollowing features. The longer videos can be full-length music videosthat include audio tracks containing full-length songs. The plurality ofpreselected videos can include audio tracks containing excerpts of thefull-length songs. An audio track for the particular video, in itsentirety, can be an excerpt of the audio track for a particular longervideo. A video track for the particular video can include (i) a firstportion that is an excerpt of a video track for the particular longervideo and (ii) one or more second portions that are filler video notfrom the particular longer video. The one or more second portions of theparticular video can be locations in the particular video wherepersonalized video tracks derived from the personal media content areautomatically inserted. The video tracks can be derived from thepersonal media content are not inserted at or over the first portion.The first portion of the video track can correspond to an emotionallyimpactful moment in the particular video. Personal media content can bedesignated as being the most emotionally impactful from among thepersonal media content is automatically positioned immediately followingthe first portion. The longer videos can be full-length movies thatinclude audio tracks containing full-length movie sound tracks. Theplurality of preselected videos can include audio tracks containingexcerpts of the full-length movie sound tracks.

The personal media content can include one or more of: digital photos,digital videos, and personalized text. The method can further includeautomatically analyzing, by the client computing device, waveforms foranother longer video to automatically identify an emotionally impactfulmoment; determining, by the client computing device, starting and endpoints within the other longer video based on intro and outro transitionpoints within a threshold timestamp from the emotionally moment in theother longer video; automatically generating, by the client computingdevice, a video excerpt from the other longer video using the startingand end points; and adding the video excerpt from the other longer videoto the plurality of preselected videos. The method can further includegenerating a personalization template for the video excerpt from theother longer video based, at least in part, on the location of theemotionally impactful moment within the video excerpt. The automaticwaveform analysis can be performed based on one or more of the followingwaveform characteristics: mode, volume, tempo, mood, tone, and pitch.The personalized video can include a mediagram that is intended toprovide an emotionally impactful message that is specifically tailoredto a relationship between a sender and recipient.

In another implementation, a method for providing a social mediaplatform for enhancing and improving social interactions among usersincludes retrieving, by a relationship concierge running on a socialmedia system, (i) user profiles for a first user and a second user, and(ii) a relationship profile for a relationship between the first userand the second user; retrieving, by the relationship concierge,historical interactions among the first user and the second user on thesocial media platform; determining, by the relationship concierge,whether provide a social interaction prompt to one or more of the firstuser and the second user based on the user profiles, the relationshipprofile, and the historical interactions, wherein the social interactionprompt provides a call to action for interaction within the relationshipbetween the first user and the second user; identifying, in response todetermining that the social interaction prompt is to be provided, thefirst user from among the first and second users as the recipient of thesocial interaction prompt; automatically transmitting, by therelationship concierge and without a request from either the first orsecond user, the social interaction prompt to a first client computingdevice for the first user, wherein the social interaction prompt is onlyvisible to the first user until the first user responds to the prompt;receiving, at the relationship concierge, a response from the socialinteraction prompt; automatically transmitting, by the relationshipconcierge and without a request from either the first or second user,the response with the social interaction prompt to a second clientcomputing device for the second user, wherein the social interactionprompt and the response are presented to the second user by the secondclient computing device based on the first user having provided theresponse.

Such an implementation can optionally include one or more of thefollowing features. The social interaction prompt can include a questionthat is posed to the first user. The social interaction prompt caninclude the first user being directed to create a mediagram for thesecond user. The mediagram can include a personalized video segment thatis automatically personalized to provide an emotionally impactfulmessage that is particularly tailored to the relationship between thefirst and second users. The social interaction prompt can include aninteractive game to be played by the first and second users. The firstand second users can interact on the social platform via a private wallthat is exclusive to the first and second users. The social interactionprompt can be initially only visible on the private wall to the firstuser. The social interaction prompt can become visible on the privatewall to the second user in only after and in combination with theresponse to the social interaction prompt by the user. The second usercan be delayed from replying to the response for at least a thresholdperiod of time following the response and the social interaction promptappearing to the second user on the private wall. The response and thesocial interaction prompt can be automatically deleted from the privatewall after a threshold amount of time or interactions have elapsed sincethey appeared on the private wall.

In another implementation, a computer-implemented method includesreceiving from a first user a selection of a sub-portion of a musicvideo that includes audio from a sub-portion of a song and video thatcorresponds to the audio; receiving from the first user personalizationcontent entered into a template that designates particular types ofmedia content to be combined with the music video at particularlocations of the music video; providing, to a second user who wasdesignated by the first user, an indication that the content isavailable for review by the second user; and providing, to the seconduser and in response to a second user confirmation of the providedindication, the sub-portion of the music video in combination with thepersonalization content.

Such an implementation can optionally include one or more of thefollowing features. The method can further include previouslydetermining portions in each of a plurality of music videos,sub-portions of each of the plurality of music videos that will have anincreased impact on a viewer of the sub-portions as compared to othersub-portions of the music videos. Determining the portions can includemanually reviewing the videos with trained human classifiers.Determining the portion can include identifying which portions ofparticular videos are played the most by visitors to one or more on-linevideo sites. Determining the portions can include performing automaticmusic analysis of a plurality of different music videos to identifymusical patterns previously determined to have an emotional effect on atypical listener. The personalization content can include a textualmessage entered by the first user. The second user can be provided withone or more bumpers created by the first user and appended to the front,back, or both of the video sub-portion.

In another implementation, a system for providing a social mediaplatform to enhance the quality of online social interactions amongusers, the system including: first and second client computing devicesthat are running social media applications for the social mediaplatform, each of the social media applications being programmed toprovide a graphical user interface (GUI) that presents digital contentretrieved over the internet from the social media platform and toreceive user inputs via one or more graphical input elements in the GUI,the first client computing being associated with a first user and thesecond client computing device being associated with a second user; adigital profile repository storing (i) user profiles for a first userand a second user, and (ii) a relationship profile for a relationshipbetween the first user and the second user; a relationship historydatabase storing historical interactions among the first user and thesecond user on the social media platform; and a relationship conciergeto facilitate meaningful social interactions among the first and secondclient computing devices, the relationship concierge being programmedto: retrieve the user profiles for the first user and the second user,and the relationship profile for the relationship between the first userand the second user from the digital profile repository, retrieve thehistorical interactions among the first user and the second user on thesocial media platform from the relationship history database, determinewhether to provide a social interaction prompt to one or more of thefirst client computing device and the second client computing devicebased on the user profiles, the relationship profile, and the historicalinteractions, wherein the social interaction prompt provides a call toaction for interaction within the relationship between the first userand the second user, identify, in response to determining that thesocial interaction prompt is to be provided, the first client computingdevice from among the first and second users as the recipient of thesocial interaction prompt, automatically transmit, without a requestfrom either the first client computing device or the second clientcomputing device, the social interaction prompt to a first clientcomputing device, wherein the social interaction prompt is onlypresented in the GUI on the first client computing device for the firstuser until the first user responds to the prompt, and the socialinteraction prompt is not transmitted to the second client computingdevice or presented in the GUI on the second client computing device,receive a response to the social interaction prompt from the firstclient computing device, and automatically transmit, without a requestfrom either the first client computing device or the second clientcomputing device, the response with the social interaction prompt to thesecond client computing device for presentation in the GUI on the secondclient computing device, wherein the social interaction prompt and theresponse are presented in the GUI on the second client computing devicebased on the first user having provided the response.

Such an implementation can optionally include one or more of thefollowing features. The social interaction prompt can include a questionthat is posed in the GUI on the first client computing device to thefirst user. The social interaction prompt can include the first userbeing directed to create a mediagram for the second user, wherein themediagram comprises a personalized digital video segment that isautomatically personalized to provide an emotionally impactful messagethat is particularly tailored to the relationship between the first andsecond users. The social interaction prompt can include an interactivegame to be played by the first and second users. The GUI on the firstclient computing device and the GUI on the second client computingdevice can provide a private wall that is exclusive to the relationshipbetween the first and second users, the social interaction prompt caninitially be only visible on the private wall presented by the firstclient computing device to the first user, and the social interactionprompt can become visible on the private wall presented by the secondclient computing device to the second user only after and in combinationwith the response to the social interaction prompt by the user. The GUIin the second client computing device can delay the second user fromreplying to the response for at least a threshold period of timefollowing the response and the social interaction prompt being presentedon the private wall of the second client computing device. The GUI inthe second client computing device (i) can inactivate the graphicalinput elements to receive a reply from the second user until after adelayed response period has elapsed, and (ii) can presents timinginformation identifying an amount of time remaining until the delayedresponse period has elapsed, and the GUI in the first client computingdevice can also present the timing information identifying an amount oftime remaining until the delayed response period has elapsed for thesecond user to respond. The GUI in the second client computing device(i) can activate the graphical input elements to receive a reply fromthe second user during a delayed response period and (ii) can presenttiming information identifying an amount of time remaining until thedelayed response period has elapsed and the reply from the second userwill be transmitted to the first client computing device, and the GUI inthe first client computing device can also present the timinginformation identifying an amount of time remaining until the delayedresponse period has elapsed for the second user's reply to betransmitted to the first client computing device. The response and thesocial interaction prompt can be automatically deleted from the privatewall after a threshold amount of time or interactions have elapsed sincethey appeared on the private wall.

In another implementation, a computer-implemented method for providing asocial media platform to enhance the quality of online socialinteractions among users, the computer-implemented method comprising:retrieving, from a digital profile repository storing (i) user profilesfor a first user and a second user, and (ii) a relationship profile fora relationship between the first user and the second user, user profilesfor the first user and the second user, and the relationship profile forthe relationship between the first user and the second user, retrieving,from a relationship history database storing historical interactionsamong the first user and the second user on the social media platform,historical interactions among the first user and the second user on thesocial media platform, and facilitating meaningful social interactionsamong the first and second client computing devices that are runningsocial media applications for the social media platform, each of thesocial media applications being programmed to provide a graphical userinterface (GUI) that presents digital content retrieved over theinternet from the social media platform and to receive user inputs viaone or more graphical input elements in the GUI, the first clientcomputing being associated with the first user and the second clientcomputing device being associated with the second user, the facilitatingincluding: determining whether to provide a social interaction prompt toone or more of the first client computing device and the second clientcomputing device based on the user profiles, the relationship profile,and the historical interactions, wherein the social interaction promptprovides a call to action for interaction within the relationshipbetween the first user and the second user, identifying, in response todetermining that the social interaction prompt is to be provided, thefirst client computing device from among the first and second users asthe recipient of the social interaction prompt, automaticallytransmitting, without a request from either the first client computingdevice or the second client computing device, the social interactionprompt to a first client computing device, wherein the socialinteraction prompt is only presented in the GUI on the first clientcomputing device for the first user until the first user responds to theprompt, and the social interaction prompt is not transmitted to thesecond client computing device or presented in the GUI on the secondclient computing device, receiving a response to the social interactionprompt from the first client computing device, and automaticallytransmitting, without a request from either the first client computingdevice or the second client computing device, the response with thesocial interaction prompt to the second client computing device forpresentation in the GUI on the second client computing device, whereinthe social interaction prompt and the response are presented in the GUIon the second client computing device based on the first user havingprovided the response.

Such an implementation can optionally include one or more of thefollowing features. The social interaction prompt can include a questionthat is posed in the GUI on the first client computing device to thefirst user. The social interaction prompt can include the first userbeing directed to create a mediagram for the second user, wherein themediagram comprises a personalized digital video segment that isautomatically personalized to provide an emotionally impactful messagethat is particularly tailored to the relationship between the first andsecond users. The social interaction prompt can include an interactivegame to be played by the first and second users. The GUI on the firstclient computing device and the GUI on the second client computingdevice can provide a private wall that is exclusive to the relationshipbetween the first and second users, the social interaction prompt caninitially be only visible on the private wall presented by the firstclient computing device to the first user, and the social interactionprompt can become visible on the private wall presented by the secondclient computing device to the second user only after and in combinationwith the response to the social interaction prompt by the user. The GUIin the second client computing device can delay the second user fromreplying to the response for at least a threshold period of timefollowing the response and the social interaction prompt being presentedon the private wall of the second client computing device. The GUI inthe second client computing device (i) can inactivate the graphicalinput elements to receive a reply from the second user until after adelayed response period has elapsed, and (ii) can presents timinginformation identifying an amount of time remaining until the delayedresponse period has elapsed, and the GUI in the first client computingdevice can also present the timing information identifying an amount oftime remaining until the delayed response period has elapsed for thesecond user to respond. The GUI in the second client computing device(i) can activate the graphical input elements to receive a reply fromthe second user during a delayed response period and (ii) can presenttiming information identifying an amount of time remaining until thedelayed response period has elapsed and the reply from the second userwill be transmitted to the first client computing device, and the GUI inthe first client computing device can also present the timinginformation identifying an amount of time remaining until the delayedresponse period has elapsed for the second user's reply to betransmitted to the first client computing device. The response and thesocial interaction prompt can be automatically deleted from the privatewall after a threshold amount of time or interactions have elapsed sincethey appeared on the private wall.

In another implementation, a non-transitory computer-readable medium forproviding a social media platform to enhance the quality of onlinesocial interactions among users and storing instructions, that whenexecuted, cause one or more processors to perform operations including:retrieving, from a digital profile repository storing (i) user profilesfor a first user and a second user, and (ii) a relationship profile fora relationship between the first user and the second user, user profilesfor the first user and the second user, and the relationship profile forthe relationship between the first user and the second user, retrieving,from a relationship history database storing historical interactionsamong the first user and the second user on the social media platform,historical interactions among the first user and the second user on thesocial media platform, and facilitating meaningful social interactionsamong the first and second client computing devices that are runningsocial media applications for the social media platform, each of thesocial media applications being programmed to provide a graphical userinterface (GUI) that presents digital content retrieved over theinternet from the social media platform and to receive user inputs viaone or more graphical input elements in the GUI, the first clientcomputing being associated with the first user and the second clientcomputing device being associated with the second user, the facilitatingincluding: determining whether to provide a social interaction prompt toone or more of the first client computing device and the second clientcomputing device based on the user profiles, the relationship profile,and the historical interactions, wherein the social interaction promptprovides a call to action for interaction within the relationshipbetween the first user and the second user, identifying, in response todetermining that the social interaction prompt is to be provided, thefirst client computing device from among the first and second users asthe recipient of the social interaction prompt, automaticallytransmitting, without a request from either the first client computingdevice or the second client computing device, the social interactionprompt to a first client computing device, wherein the socialinteraction prompt is only presented in the GUI on the first clientcomputing device for the first user until the first user responds to theprompt, and the social interaction prompt is not transmitted to thesecond client computing device or presented in the GUI on the secondclient computing device, receiving a response to the social interactionprompt from the first client computing device, and automaticallytransmitting, without a request from either the first client computingdevice or the second client computing device, the response with thesocial interaction prompt to the second client computing device forpresentation in the GUI on the second client computing device, whereinthe social interaction prompt and the response are presented in the GUIon the second client computing device based on the first user havingprovided the response.

Such an implementation can optionally include one or more of thefollowing features. The social interaction prompt can include a questionthat is posed in the GUI on the first client computing device to thefirst user.

In another implementation, a system for generating digital media filesincludes a digital media file repository, a front end system, a backendsystem, and a digital media distribution system. The digital media filerepository stores a plurality of preselected digital video files thatare excerpts of longer digital video files. The plurality of preselecteddigital video files are encoded in a common digital video codec and arestored with metadata that identifies times within the plurality ofpreselected digital video files at which emotionally impactful momentsoccur.

The frontend system is in communication with client computing devices.The frontend system receives digital media file content generationrequests from the client computing devices that include parametersidentifying particular preselected digital video files to be combinedwith personal digital media files to generate personalized digital videofiles. The personal digital media files include personal digital videofiles, personal digital audio files, personal text, and personal digitalimage files that are uploaded to the frontend system by the clientcomputing devices. the personal digital video files are encoded across aplurality of digital video codecs.

The backend system generates the personalized digital video files usingthe particular preselected digital video files and the personal digitalmedia files. The backend system being programmed to: convert thepersonal digital video files from the plurality of digital video codecsto the common video codec; retrieve personalization digital mediatemplates that designate (i) particular types of media content to becombined with the particular preselected digital video files and (ii)particular times within particular preselected digital video files atwhich the particular types of media content are to be combined with theparticular preselected digital video files, the particular times beingrelative to the times within the plurality of preselected digital videofiles at which the emotionally impactful moments occur; assemble digitalmedia content for the personalized digital video files using particularpreselected digital video files, the digital media templates, and thepersonal digital media files, the personal digital media files being (i)positioned at the particular times relative to the times at which theemotionally impactful moments occur in the particular preselecteddigital video files, (ii) visually combined with video tracks of theparticular preselected digital video files so that digital images andvideos from the personal digital media files replace the video tracks atthe particular times, and (iii) audibly combined with audio tracks forthe particular preselected digital video files so that audio from thepersonal digital media files are automatically mixed with the audiotracks at the particular times, wherein the video tracks and the audiotracks for the particular preselected digital video files at the timesat which the emotionally impactful moments occur remain unmodified inthe assembled digital media content; encode the assembled digital mediacontent using the common video codec to generate the personalizeddigital video files; and store the personalized digital video files. Thedigital media distribution system is configured to transmit thepersonalized digital video files to the client computing devices.

Such an implementation can optionally include one or more of thefollowing features. The longer digital video files can be full-lengthmusic videos containing full-length songs and the plurality ofpreselected digital video files can be excerpts of the full-length musicvideos that include the emotionally impactful moments. The longerdigital video files can be full-length movies and the plurality ofpreselected digital video files can be excerpts of the full-lengthmovies that include the emotionally impactful moments. The personalizeddigital video files can comprise mediagrams that include a personalizedmessage centered around the emotionally impactful moments in theparticular preselected digital video files and the mediagrams can beconfigured to be digitally sent from one client computing device toanother client computing device. The personal digital media files canhave variable lengths of time. Assembling the digital media content caninclude adding one or more portions of digital filler content so as (i)to fit the personal digital media files with variable lengths of time atthe particular times according to the digital media templates and (ii)ensure that the video tracks and the audio tracks for the particularpreselected digital video files at the times at which the emotionallyimpactful moments occur remain unmodified in the assembled digital mediacontent. The one or more portions of digital filler content can be loopsof digital content derived from the particular preselected digital videofiles. The one or more portions of digital filler content can bepreselected loops of digital content.

In another implementation, a computer-implemented method includesreceiving digital media file content generation requests from clientcomputing devices that include parameters identifying particularpreselected digital video files to be combined with personal digitalmedia files to generate personalized digital video files, the personaldigital media files including personal digital video files, personaldigital audio files, personal text, and personal digital image files,the personal digital video files being encoded across a plurality ofdigital video codecs, the preselected digital video files being excerptsof longer digital video files, the preselected digital video files beingencoded in the common digital video codec and being stored with metadatathat identifies times within the preselected digital video files atwhich emotionally impactful moments occur. The method further includesconverting the personal digital video files from the plurality ofdigital video codecs to a common video codec. The method furtherincludes retrieving personalization digital media templates thatdesignate (i) particular types of media content to be combined with theparticular preselected digital video files and (ii) particular timeswithin particular preselected digital video files at which theparticular types of media content are to be combined with the particularpreselected digital video files, the particular times being relative tothe times within the plurality of preselected digital video files atwhich the emotionally impactful moments occur. The method furtherincludes assembling digital media content for the personalized digitalvideo files using particular preselected digital video files, thedigital media templates, and the personal digital media files, thepersonal digital media files being (i) positioned at the particulartimes relative to the times at which the emotionally impactful momentsoccur in the particular preselected digital video files, (ii) visuallycombined with video tracks of the particular preselected digital videofiles so that digital images and videos from the personal digital mediafiles replace the video tracks at the particular times, and (iii)audibly combined with audio tracks for the particular preselecteddigital video files so that audio from the personal digital media filesare automatically mixed with the audio tracks at the particular times,wherein the video tracks and the audio tracks for the particularpreselected digital video files at the times at which the emotionallyimpactful moments occur remain unmodified in the assembled digital mediacontent. The method further includes encoding the assembled digitalmedia content using the common video codec to generate the personalizeddigital video files. The method further includes storing thepersonalized digital video files. The method further includestransmitting the personalized digital video files to the clientcomputing devices.

Such an implementation can optionally include one or more of thefollowing features. The longer digital video files can be full-lengthmusic videos containing full-length songs and the plurality ofpreselected digital video files can be excerpts of the full-length musicvideos that include the emotionally impactful moments. The longerdigital video files can be full-length movies and the plurality ofpreselected digital video files can be excerpts of the full-lengthmovies that include the emotionally impactful moments. The personalizeddigital video files can comprise mediagrams that include a personalizedmessage centered around the emotionally impactful moments in theparticular preselected digital video files and the mediagrams can beconfigured to be digitally sent from one client computing device toanother client computing device. The personal digital media files canhave variable lengths of time. Assembling the digital media content caninclude adding one or more portions of digital filler content so as (i)to fit the personal digital media files with variable lengths of time atthe particular times according to the digital media templates and (ii)ensure that the video tracks and the audio tracks for the particularpreselected digital video files at the times at which the emotionallyimpactful moments occur remain unmodified in the assembled digital mediacontent. The one or more portions of digital filler content can be loopsof digital content derived from the particular preselected digital videofiles. The one or more portions of digital filler content can bepreselected loops of digital content.

In another implementation, a computer program product encoded on anon-transitory storage medium comprises non-transitory, computerreadable instructions for causing one or more processors to performoperations. The operations include receiving digital media file contentgeneration requests from client computing devices that includeparameters identifying particular preselected digital video files to becombined with personal digital media files to generate personalizeddigital video files, the personal digital media files including personaldigital video files, personal digital audio files, personal text, andpersonal digital image files, the personal digital video files beingencoded across a plurality of digital video codecs, the preselecteddigital video files being excerpts of longer digital video files, thepreselected digital video files being encoded in the common digitalvideo codec and being stored with metadata that identifies times withinthe preselected digital video files at which emotionally impactfulmoments occur. The operations further include converting the personaldigital video files from the plurality of digital video codecs to acommon video codec. The operations further include retrievingpersonalization digital media templates that designate (i) particulartypes of media content to be combined with the particular preselecteddigital video files and (ii) particular times within particularpreselected digital video files at which the particular types of mediacontent are to be combined with the particular preselected digital videofiles, the particular times being relative to the times within theplurality of preselected digital video files at which the emotionallyimpactful moments occur. The operations further include assemblingdigital media content for the personalized digital video files usingparticular preselected digital video files, the digital media templates,and the personal digital media files, the personal digital media filesbeing (i) positioned at the particular times relative to the times atwhich the emotionally impactful moments occur in the particularpreselected digital video files, (ii) visually combined with videotracks of the particular preselected digital video files so that digitalimages and videos from the personal digital media files replace thevideo tracks at the particular times, and (iii) audibly combined withaudio tracks for the particular preselected digital video files so thataudio from the personal digital media files are automatically mixed withthe audio tracks at the particular times, wherein the video tracks andthe audio tracks for the particular preselected digital video files atthe times at which the emotionally impactful moments occur remainunmodified in the assembled digital media content. The operationsfurther include encoding the assembled digital media content using thecommon video codec to generate the personalized digital video files. Theoperations further include storing the personalized digital video files.The operations further include transmitting the personalized digitalvideo files to the client computing devices.

Such an implementation can optionally include one or more of thefollowing features. The longer digital video files can be full-lengthmusic videos containing full-length songs and the plurality ofpreselected digital video files can be excerpts of the full-length musicvideos that include the emotionally impactful moments. The longerdigital video files can be full-length movies and the plurality ofpreselected digital video files can be excerpts of the full-lengthmovies that include the emotionally impactful moments. The personalizeddigital video files can comprise mediagrams that include a personalizedmessage centered around the emotionally impactful moments in theparticular preselected digital video files and the mediagrams can beconfigured to be digitally sent from one client computing device toanother client computing device. The personal digital media files canhave variable lengths of time. Assembling the digital media content caninclude adding one or more portions of digital filler content so as (i)to fit the personal digital media files with variable lengths of time atthe particular times according to the digital media templates and (ii)ensure that the video tracks and the audio tracks for the particularpreselected digital video files at the times at which the emotionallyimpactful moments occur remain unmodified in the assembled digital mediacontent. The one or more portions of digital filler content can be loopsof digital content derived from the particular preselected digital videofiles. The one or more portions of digital filler content can bepreselected loops of digital content.

Particular implementations may realize none, one or more of thefollowing advantages. For example, media content can be personalized inways that ensure that synchronization between audio and video portionsof the underlying media content are not disrupted. When editing mediacontent, particularly when overlaying video and/or audio tracks acrossdifferent devices, video and audio can get out of sync.

In another example, media content personalization can be streamlined toprovide novice users with the ability to readily create impactfulpersonalized content. In particular, a user interface can be presentedto users that narrows the field of options for personalization down to alimited number through the use of preselected media content excerpts,personalization templates, guided personalization steps, and otherfeatures to ensure emotionally impactful personalized content iscreated.

In another example, social platforms can facilitate improved and moremeaningful social interactions and relationships between users through avariety of features, such as private walls, relationship concierges,time delays for interactions, personalized media content distribution,time-limited social content, and/or combinations thereof. For instance,given the open nature of many social platforms (meaning that posts areviewable to a broad audience of users), social interactions can beguarded and reserved. By providing a social platform in which a primarymechanism for interacting with other users is either private walls orprivate group walls, user interactions can be with smaller and moreintimate sets of users, which can help users drop their guard andinteract more naturally/honestly. In another instance, relationshipconcierges can automate and assist users in building and maintainingstrong relationships by prompting users with ways to interact with eachother. In a further instance, personalized media content creation anddistribution on a social platform can assist users in conveyingemotionally impactful messages that may otherwise be difficult toexpress through traditional social media interactions (e.g., posts,images, text). In another instance, mandatory time delays forinteractions between users can alleviate the pressure, stress, andburden that users feel to promptly respond to interactions in order toavoid expressing disinterest with a late response or no response at all.In a further instance, time-limited social content can additionallypromote more natural/honest social interactions (help users drop theirguard) by ensuring that social interactions on the platform will notpersist for perpetuity, but instead will be inaccessible to both usersafter a period of time or after a series of interactions. The app canforce both senders and recipient to be reflective based on the appforcing them to have a delay before sending/receiving the message. Thetime delays introduce a component of “scarcity” which enforcesreflection, anticipation and attention to detail, fostering betterrelationships.

The details of one or more implementations of the subject matterdescribed in this specification are set forth in the accompanyingdrawings and the description below. Other features, aspects, andadvantages of the subject matter will become apparent from thedescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example system for generatingpersonalized media content.

FIG. 2 is a block diagram showing an example technique for creating anddelivering a personalized mediagram to a recipient.

FIG. 3 is a block diagram of an example system for generating andconsuming mediagrams.

FIGS. 4A-F are screenshots that collectively show an example sequence ofsteps for creating and distributing a mediagram.

FIGS. 5A-M are block diagrams showing example assemblies of mediagrams.

FIG. 6 is a conceptual diagram of an example system for generatingpersonalized media content.

FIGS. 7A-B are a flowchart of an example technique for generatingpersonalized videos.

FIG. 8A is a conceptual diagram of an example social media platform forproviding improved and more meaningful social interactions among users.

FIG. 8B is a conceptual diagram of another example social media platformfor providing improved and more meaningful social interactions amongusers.

FIG. 9A is an example system for providing an improved social mediaplatform with more meaningful social interactions among users.

FIG. 9B is diagram of an example system for providing an improved socialmedia platform with more meaningful social interactions among users.

FIG. 9C depicts an example system for providing an improved social mediaplatform with more meaningful social interactions among users.

FIG. 10 is a flow chart with user interfaces and to establish an initialconnection between users on a social media platform.

FIGS. 11A-B are screenshots of example user interfaces on an examplemobile computing device for interacting with other users via privatewalls on a social platform.

FIGS. 11C-F present example specific user interface features that can beselected for presentation to users.

FIGS. 12A-H are screenshots of an example process flow for arelationship concierge facilitating and improving social interactionsamong users via private walls on a social platform.

FIGS. 13A-C is screenshot of an example user interface on a mobilecomputing device for viewing a user's friends and the correspondinginteraction delays until another relationship concierge prompt isexpected.

FIG. 14A is a conceptual diagram of an example personal concierge systemand algorithm for facilitating and improving user relationships on asocial network.

FIG. 14B is a diagram of an example system to vary content that isselected for presentation to users.

FIG. 14C is a screenshot of an example “one-click” feedback interface inwhich content is presented with selectable graphical elements that theuser can select with a single click/selection action to provide feedbackrelated to the content.

FIGS. 15A-D are screenshots of a relationship concierge being applied toother social platforms providing predominantly open communication amongbroad groups of users.

FIG. 16 is a diagram depicting creation and use of a private group wallon a social platform to improve and enhance meaningful socialinteractions.

FIGS. 17A-H are screenshots of an example user interface on a computingdevice for users to express and interact with others regarding theiremotional well-being.

FIGS. 18A-B are flowcharts of example techniques for determining andtransmitting prompts to specific relationship private walls on a socialplatform.

FIG. 19 is a flowchart of an example technique for determining andtransmitting delays between interactions on a social platform.

FIG. 20 is a flowchart of an example technique for determiningrelationship ratings on a social platform.

FIGS. 21A-B are flowcharts of example techniques for creating and usingprivate group walls on a social platform.

FIG. 22 is a block diagram of example computing devices.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of an example system 100 for generatingpersonalized media content, such as mediagrams. A mediagram can bepersonalized media content that is configured in a particular manner toconvey an emotionally impactful message between users. Mediagrams caninclude, for example, underlying media content that is combined withother, personal media content to provide personalization to theunderlying media content. Media content can include, for instance, musicand/or video excerpts, movie excerpts, music files, images, televisionclips (ex. SNL skits), Viral videos (ex. Home videos that have beenpopularized), concert videos, and/or other types of media. For example,a mediagram can be an excerpt from a music video that is personalizedwith text, images, audio, and video.

A mediagram that is produced by the system 100 can be a ready-to-playpresentation of media that is prepared by a sender 102 and sent to atleast one recipient 104. The mediagram can include, for example, musicfiles and other media that are combined into the mediagram in a way thatis personalized by the sender 102. Personalization can include addingpersonalized messages and/or other elements to the media, includingtext, audio, images, video, etc., which can overlay and/or adjoinsegments of the media. For example, personalization can be a captionthat precedes or accompanies an image, a video, or some other mediasegment.

As an example, the system 100 can provide, for presentation to thesender 102, different media segments that can be selected by the sender102 for personalization. For example, the sender 102 can select fromamong media content 106 a-106 d (e.g., music videos, music, movies,videos). The media content 106 a-106 b may be presented to the sender102, for example, upon execution of a search query (e.g., to find songsfor specific artists, titles, subjects, genres, etc.). Selection ofmedia content can be made, for example, from the sender's library ofdownloaded and/or owned media content, generated from a subscribed listof available songs, and/or in some other way. Presentation of the mediacontent 106 a-106 d, as well as other aspects of a user interface forcreating mediagrams, can be presented on a user device 107 of the sender102.

In some implementations, media content and/or excerpts of media content(e.g., portions of a song, portions of a movie, portions of a musicvideo) can be pre-selected to provide a chill moment, which can be apoint in a song or other media that has been shown to provide a chill toa viewer of the media (e.g., moment that provides tingles, a chillrunning down one's spine, a significant emotional and/or physicalresponse, or some other reaction by the recipient 104). Example chillmoments in songs include a particular note or passage in a song,particular lyrics, and/or other features that are otherwise emotionallyimpactful upon users. Example chill moments in a movie or video caninclude a chase scene or an important scene, such as a celebratorymoment, the death of a character, or some other significantly impactfulsegment.

In some implementations, media content with chill moments can beidentified in a catalog of chill moments, which may be stored in (andavailable from) a proprietary catalog. Each chill moment, identified fora particular song or movie, for example, can identify a point in theparticular song or movie that produces a “chill” reaction (in anaudience of the media) that results in a sensation that is similar tofeeling cold, getting goosebumps, having one's hair stand on end, orsome other physiological reaction. Such reactions can include, forexample, an increased heart rate, an increased respiration rate, inincreased skin conductance (or “galvanic skin response”), or some otherphysiological response. Sources for chills can include, for example,music (e.g., most potent), visual arts, speeches (e.g., notablespeechmakers), beauty (or other breath-taking appearance), or physicalcontact. The intensity and/or effect of chills can be affected byfactors such as mode, volume, tempo, mood, tone, and pitch, or otherfactors that may help to convey or amplify emotion. Generally, it can beassumed that chill moments can only exist for music or video that havealready been experienced before, such as by at least one user.

In the depicted example, the sender 102 select the media content 106 cfrom among the media content 106 a-106 d. While the selected mediacontent 106 c can refer to the entire media content (e.g., the entiresong, the entire movie, the entire music video), the sender 102 canselect from among media content excerpts 106 c′-106 c′″ (segments of themedia content 106), each of which may include the chill moment that thesender 102 wishes to share with the recipient 104. The excerpts 106c′-106 c′″ can be pre-designated for the media content 106 c (e.g.,manually designated, crowd sourced) and/or automatically identified(e.g., waveform analysis). Each of the other media content 106 a-b and106 d can additionally include one or more excerpts that are proposed tothe sender 102 for selection.

In the depicted example, the sender 102 selects the excerpt 106 c′ forpersonalization. However, at this point the media content excerpt 106 c′has yet to be personalized (e.g., just a segment of a music videowithout personalization). The sender 102 can be prompted to identifypersonal media content to add to the selected excerpt 106 c′, such asoverlaying a portion of the excerpt 106 c′ (e.g., photo overlaying aportion of a music video, text overlaying a portion of a movie), beingpresented adjacent to the selected excerpt 106 c′ (e.g., video that isplayed before or after the excerpt 106 c′), and/or other combinationswith the excerpt 106 c′. The sender 102 can be guided through selectionof media content for the excerpt 106 c′ by the sender's device 107,which can be programmed, for example, to use one or more personalizationtemplates to assist the sender 102 in the selection of personalizedmedia content to provide maximize the emotional impact of thepersonalized media content. For example, the sender 102 can be promptedto enter a textual message for the recipient 104, then to provide up to10 seconds of a personalized video message to the recipient 104, andthen to provide up to 3 photos that include both the sender 102 and therecipient 104. In the depicted example, the sender 102 selects theexample personal media content 108 (e.g., photos, videos, audio, text)to be used to personalize the excerpt 106 c′ for the mediagram.

A server system 112 (e.g., cloud computer system) can receive theselection of the media content excerpt 106 c′ along with the personalmedia content 108 and can generate a mediagram to be delivered to therecipient 104. Such generation can include, for example, referencing oneor more personalization templates to determine how to combine theexcerpt 106 c′ with the personal media content 108 (as well asreferencing particular instructions/designations for the mediagram madeby the sender 102). The generation can also be completed using digitalrights management code, which can be encoded into the resultingmediagram to manage aspects of copyrights and payment of royaltiesand/or other fees. The mediagram can be output, for example, in the formof deliverable media 110 (e.g., video file, audio file) that resultsfrom audio, video, images, text, and/or other media content items beingassembled by a server 112. For example, the deliverable media 110 can bea single video file and that is transmitted to a computing device 114for the recipient 104.

In some implementations, the component parts of the mediagram, includingthe associated media segments, can be sent individually (e.g., not inthe form of deliverable media 110) and assembled by the computing device114 for presentation to the recipient 104.

The deliverable media 110 can be provided to the device 114 using one ormore features to protect against piracy. For example, the deliverablemedia 110 can be provided in a “lock box” to protect media and avoidpiracy. The lock box may include a feature that prevents consumption ofthe mediagram unless the recipient 104 provides credentials or someother form of authentication. In another example, the mediagram caninclude features that control the number of times and/or a timeframeover which the mediagram is presented, such as a single time or alimited number of times, or an expiration time that limits presentationof the mediagram to a time-limited viewing. In another example, themediagram deliverable 110 can include digital rights management (DRM)features and/or other techniques for copyright protection of digitalmedia, including restricting copying of media and preventingunauthorized redistribution. In another example, the device 114 of therecipient 104 may be required to install/run/load aspecialized/authorized media player (or an application providing similarfunctionality) to view content of the mediagram.

The mediagram deliverable 110 can be distributed to the device 114 inany of a variety of ways, such through an account that the recipient 104may have on the server system 112 (e.g., push notification provided tomobile app on the device 114 that is hosted by the server system 112),by transmitting a link to the deliverable 110 (e.g., sending an emailincluding a uniform resource locator (URL) for the deliverable 110,sending a text message including the URL for the deliverable). Otherways of providing notification to the recipient 104 that the deliverable110 is available and ready for him/her to access it are also possible.

FIG. 2 is a block diagram 200 showing an example technique 202 forcreating and delivering a personalized mediagram 204 to a recipient. Inthe depicted example, the mediagram 204 is created using a music videoas the underlying media content that is being personalized.

To create the mediagram 204, a user can starts by selecting a song thatwill be personalized (206). For example, referring to FIG. 1, the sender102 selects a song from pre-made categories or uses a search feature toidentify songs by artist, song title, or occasion/subject (e.g.,Christmas songs, love songs, etc.), or in some other way. The user canthen create a personal message through personal media content that theuser selects (208). For example, referring to FIG. 1, the sender 102 candesignate text, audio, photos, videos, and/or other personal mediacontent to be added and/or otherwise combined with the selected song togenerate the mediagram 204. As described above with regard to FIG. 1,the user can be guided through the selection and designation of thepersonal media content for the mediagram, such as through the use ofpersonalization templates that can identify specific types of mediacontent that should be added to particular locations of the song excerptto maximize the emotional impact of the mediagram. Additionally and/oralternatively, for senders and receiver pairs who have relationshipsmodeled by a relationship concierge (e.g., users of the social platformsdescribed below with regard to FIGS. 8-21 that use relationshipconcierges to facilitate and improve social interactions), therelationship concierge can be used to identify and select personal mediacontent for the mediagram. The relationship concierge can be used aloneand/or in combination with other features guiding personal media contentselection for the mediagram, such as the personalization temples, withthe systems, techniques, and devices described throughout this document.

For instance, the example mediagram 204 can include personalization thatis added to an original music video 222 for the song selected by theuser (206). The mediagram 204 may be for the entire music video 222 orjust a portion of the music video 22, such as an excerpt of the musicvideo 222 that has a chill moment in the song as its focal point. Themediagram 204 includes audio and video tracks, one or both of which canbe personalized by the user at various points in the mediagram 204. Inthis example, the original audio track 220 (not personalized) from themusic video run the entire length of the mediagram 204, but the originalvideo track (not personalized) for the music video 222 runs for only themiddle portion 216 of the mediagram 204. The video tracks for thebeginning 214 and end 218 of the mediagram 204 in this example arepersonalized with personal media content designated by the user. Forexample, the video track for the beginning portion 214 of the mediagram204 includes a written message and a video 214 a, and the video trackfor the end portion 218 of the mediagram 204 includes photos 218.Although the original audio 220 for the music video 222 runs the entirelength of the mediagram 204, it is combined (blended) with personalizedaudio 214 b that corresponds to the personalized video 214 a at thebeginning of the mediagram 204. The user can be guided through theprocess of selecting personal media content (214 a-b, 218) forpersonalizing the music video 222, such as with a personalizationtemplate to assist the user in identifying the best type of mediacontent to select to make the mediagram 204 emotionally impactful uponthe recipient.

Although not made explicit in the technique 202, once the personalmessage (214 a-b, 218) has been obtained from the user, the mediagram204 can be automatically assembled so to generate a high-qualitymediagram deliverable that combines the personal media content with theoriginal music video 222. These steps can be performed automatically bya computing device (e.g., client computing device, server system)without the user having to designate how the original or personal mediacontent should be assembled, let alone go through the process of layingout audio and video tracks for the mediagram 204. Additionally, thepersonal media content (214 a-b, 218) can be automatically positioned ator around the chill moment in the music video 222 so that the mediagram204 will be emotionally impactful for the recipient with regard to thesender and the relationship between the sender and recipient.

The assembly of the mediagram 204 with original and personal mediacontent is one example. Other configurations and arrangements ofpersonal media content with regard to original media content are alsopossible.

Once the mediagram 204 has been created, it can be sent to the recipient(210), such as by specifying the recipient's contact information (phonenumber, email, social network identifier, etc.). Once specified, themediagram 204 can be delivered either directly (e.g., file transmission)or indirectly (e.g., link transmission, notification) to the recipientalong one or more communication channels (212), such as in-appcommunications, email, or text message. Depending on the deliverymethod, the recipient can be prompted to send a response (e.g., via asocial platform), download an application (to render the mediagram), orto subscribe to a mediagram service, some or all of which may be free orhave a cost to the recipient and/or sender.

FIG. 3 is a block diagram of an example system 300 for generating andconsuming mediagrams. The system 300 can host a mediagram creation anddeliver service that can provide services to users, like the generation,storage, and distribution of mediagrams that are created by the users.The system 300 can use license agreements 302 that are held withlicensors 304, such as music, movie, and other media content copyrightowners. When used for generating and consuming mediagrams, for example,the system 300 can refer as needed to the license agreements 302 inorder to remain protected under copyright and other restrictions andlaws associated with the owners who license content to be used inmediagrams.

License agreements with copyright holders can dictate what media contentis provided by the mediagram server 308 for users to incorporate intotheir mediagrams. A media management system 306 can be dedicated toensuring that all of the original media content in a library used by themediagram server 308 is currently licensed with the licensors A-C (304a-c). The media management system 306 can maintain the licenseagreements 302 and a library of licensed original media content thathave been downloaded from the licensors 304 a-c. The media managementsystem 306 can additionally generate licensed media content excerptsthat include chill moments and provide them to the mediagram server 308,which can store it in a repository 312 a of licensed media content. Overtime, as some media content falls out of license, the media managementsystem 306 can purge unlicensed media content from the media filestorage 312 a that is used by the mediagram server 308.

A mediagram server 308 can store and manage mediagrams generated byusers 310, who may subscribe to or otherwise be enrolled with themediagram service. The mediagram server 308 can provide a server systemthrough which users can create and distribute personalized mediagramsusing preselected and licensed content from the media management system306. For example, the mediagram server 308 can store excerpts of songs,videos and other content obtained from the media management system 306.Using the mediagram server 308, for example, users 310 can requestand/or access new music and other content from the media managementsystem 306, and use the content to create mediagrams. The users 310 canalso stream and/or distribute their mediagrams at any time. Themediagram server 308 can include tools, including templates, that allowa user who is not proficient in media editing to easily createmediagrams

The process of generating a mediagram can include identifying chillmoments, “hooks,” and/or other features in the media content (e.g.,punchline of a joke) that elicit one or more target emotional responsesin a user, and then generating excerpts that include the chill moments,hooks, or other features eliciting emotional responses in users. A hookis a musical idea, often a short riff, passage, or phrase, that is usedin popular music to make a song appealing and to “catch the ear of thelistener.” The term “hook” generally applies to popular music,especially rock, R&B, hip hop, dance, and pop. In some implementations,chill moment identification can be done automatically using an algorithmbased on a variety of different factors, such as changes in tempo, mode,volume, mood, tone, pitch, etc. Chill moment identification can also bedone using crowd-sourced identification of popular moments in songs,such as using information from previous user selections of (oridentification of) favorite song parts. Identification can be donemanually by trained professionals. Hook identification and otheremotion-eliciting feature identification can be performed in similarways and can additionally and/or alternatively be used to generateexcerpts in the systems, techniques, and devices described throughoutthis document.

As shown in an expanded view of media management system 306, storagecontainers 312 can store media and associated files for use ingenerating mediagrams. For example, media file storage 312 a can storethe actual media used in the mediagrams (e.g., media content excerptswith chill moments), including media that is obtained from the mediamanagement system 306. Custom user content storage 312 b can storepersonalizations that users have added to their mediagrams and/or thefinalized mediagrams themselves.

Template storage 312 c can store templates that can be used by userscreating mediagrams, which can simplify the process of creatingmediagrams and allow users having little or no experience in combiningmedia to nonetheless generate mediagrams. Templates can identify and/orsuggest (to the user) specific types of personalization to insert atvarious points in media, and suggestions on what to insert for differentmedia content categories. Templates can be specific to media files,meaning that each media file (e.g., song, video, etc.) can have one ormore templates that help coordinate a user-identified personalization tothe specific chill moment in the media so that the chill moment is themost impactful/powerful. Additional templates can exist for differentcategories, such as romance, celebration, birthday, or other categories.

Filler media storage 312 d, can include snippets or other lengths ofaudio/visual media that can be inserted during some or all of thepersonalization to accommodate variable length personalized content. Forexample, the media content excerpts that are used to generate mediagramscan be centered around chill moments in the media content excerpts,which can be designed to occur in a middle to middle-end portion of theexcerpt. Accordingly, changing the length of a mediagram to accommodatevariation on the length of the personal content to be added to theexcerpt can throw off the positioning of the chill moment in themediagram. Fillers can be added to the beginning and/or the end of theexcerpt to accommodate for variation in the length of the mediagram dueto variable personal content without disrupting the positioning of thechill moment within the mediagram. Filler content can be looped portions(e.g., one or two bars) of instrumentals in musical content, musicalcontent with lyrics, or a few seconds of video content. Filler contentcan include copyright free content that matches up well with the mediacontent.

The mediagram server 308 can include web servers, various data stores,and services for mediagram instances 314, which can be extensible andscalable (e.g., cloud computer system). For example, web servers 315 canprovide access to external entities that access mediagrams. A mediametadata data store 314 a can include metadata associated with mediastored in the media file storage 312 a. A user account database 314 bcan identify users to the mediagram system and the users' accountinformation. A mediagram detail database 314 c can include definitionsof mediagrams that have been generated by users.

The mediagram server 308 can include at least one cloud-based server316, such as implemented or provided by Amazon Elastic Load Balancers(ELB), that distributes mediagrams (or provides access) to usercomputing devices 324. For example, user computing devices 324 that aremobile devices can be used by recipients 104 to receive distributedmediagrams, such as through one or more applications that reside on amobile device. Desktop implementations of the user computing devices324, for example, can access mediagram through a front end 318,implemented as Amazon Web Services (AWS). User computing devices 324 ofusers 310 can be used by both the sender and/or a recipient of amediagram.

FIGS. 4A-F are screenshots that collectively show an example sequence ofsteps for creating and distributing a mediagram. For example, FIGS. 4A-Eshow screenshots captured during use of a mobile application forcreating a mediagram, which can also be done through a web applicationthrough a web browser, and FIG. 4E shows an example mediagram presentedin a social network.

FIG. 4A shows an initial display of an example interface 400 used toidentify a media excerpt to use for a mediagram. For example, using asearch control 402, a user can search by song, album, artist, or inother ways. Further, controls 404 can be used to browse songs of variouscategories, such as a romance category 406 that includes songs relatedto romance. The songs (or other media) that are searchable in theinterface 400 can include songs (or other media) stored in the mediamanagement system 306. In the depicted example, the user selects theromance category 406.

FIG. 4B shows a display of the example interface 400 that presents mediaexcerpts 410 that are in the romance category 406. Each of the mediaexcerpts 410 can include a chill moment that they are intended toleverage to be emotionally impactful as part of a mediagram. In thedepicted example, the media excerpt 408 for a Taylor Swift song isselected from a list 410. Similar or different types of lists can bepresented when the search control 402 is used.

FIG. 4C shows a display of the example interface 400 in which the usercan preview the song. For example, using a preview control 411, the usercan preview the media excerpt 408. If the user then decides to use themedia excerpt 408 (e.g., Taylor Swift song) in a mediagram, then acreate control 412 can be used to initially populate a new mediagramwith the selected song.

FIG. 4D shows a display of the example interface 400 in which the usercan personalize the mediagram. For example, using various controls 414,the user can add text, photos, videos, and/or other types of personalmedia to the mediagram. Selecting a particular one of the controls 414,for example, can result in the user being guided through selection,using a template, that is specific to the media excerpt 408 (e.g.,Taylor Swift song) and/or the romance category 406. As an example, thetemplate may suggest that the user obtain a photo of the user with therecipient, then include a ten-second personalized message or textexpressing the user's feelings. The system can then automatically insertthe photo and personalized message in the right locations relative tothe media excerpt 408, generating a personalized mediagram withoutrequiring user to have media editing knowledge or skills. For example,there is no need for the user to figure out where the photo andpersonalized message should go (e.g., relative to the chill moment), howto edit video/audio, or how to perform other tasks. As such,automatically inserting the photo and personalized message can create acompilation of a professional-looking video, with complicated details ofvideo editing being handled automatically for the user. Additionalcontrols in the interface 400 can allow the user to preview and view themediagram once completed.

FIG. 4E shows a display of the example interface 400 in which the useris distributing the mediagram. In this example, the user is sending themediagram by email, but other distribution channels, including sendingvia social media, are available through controls 414. The user canselect the recipients of the user's mediagram from a contact list 416.Contacts in the contact list 416 can be annotated, such as todifferentiate between contacts who have mediagram accounts (e.g.,designated with mobile phone/app icons 420) and other contacts who donot have mediagram accounts (e.g., designated by grayed out a respectiveuser icon). If a particular recipient has a mediagram account, then themediagram can be delivered via their account. Otherwise, if a particularrecipient does not have a mediagram account, then the mediagram can bedelivered via available contact options (e.g., email, text, socialnetwork, etc.).

In some implementations, the user can optionally elect to send a giftwith the mediagram. For example, selection of a respective control fromcontrols 422 can lead to additional user interface elements that allowthe user to designate a monetary amount of or a selection of a gift thatis included (e.g., using a link or an attachment) with the mediagram.Gifts can also be integrated into the mediagram, such as with a linkthat can navigate the recipient to a web page or other resource fromwhich the gift can be redeemed. The user is also given the ability todownload and/or purchase the song or video for their personal use.

FIG. 4F shows a display of the example mediagram 424 displayed on amobile device of a recipient. For example, the mobile device can presenta mediagram entry 426 on a social network page of a User B for themediagram 424 created by a User A, as indicated by a mediagram socialnetwork header entry 428. The mediagram entry 426 can be generated, forexample, if the user selected a social network control from the controls414 in order to share the mediagram with one or more recipients who arefriends of the user in the social network. A mediagram song title 430can identify the media excerpt 408 (e.g., Taylor Swift song) selected bythe user. A mediagram description 432 can include, for example, a numberof underlined portions that are links to the content, which can helpdrive cross-user promotion based on the mediagram content. Theunderlined portions can include, for example, an artist link 432 a, asong link 432 b for the media excerpt 408, an album link 432 c, a giftlink 432 d (e.g., if a gift was selected using controls 422), and an applink 432 e by which a mediagram application can be downloaded.

FIGS. 5A-M are block diagrams showing example assemblies of mediagrams.The block diagrams depicted in FIGS. 5A-M present solutions to varioustechnical problems for mediagram creation. First, a variable amount ofuser-supplied personalized content (photos, video, text) the length ofthe audio portion of the license music video playing prior to and afterthe primary licensed video clip can present a problem for presenting achill moment at the right time within a mediagram. Audio pre-rolls canbe used to account and solve for this. The goal is to provide an audiooverlay for the user supplied personalized content that transitionsseamlessly into and out of the licensed video clip. Several of the blockdiagrams depicted in FIGS. 5A-M use pre-rolls to solve for variablepersonalized content.

Second, video and audio tracks for original content can be licensedtogether or separately, depending on contractual agreements with variouslicensors, which can create technical hurdles in generating a mediagramfile that is compliant with licensing agreements. For example, someagreements may permit the audio track from a video (e.g., movie, musicvideo) to be licensed separately (and at a lower price point) than theprice for licensing the audio and video together. However, someagreements may not permit such bifurcation of audio and video licensingrights. Additionally, some agreements may grant licenses to master audioloops from a song that could be used for pre-roll fillers, but someagreements may not. Some agreements may also grant licenses to lead inor out of the licensed media content with other content (e.g., thirdparty content, in-house generated content), but others may not. Theblock diagrams depicted in FIGS. 5A-M provide a variety of approachesand file formats for generating mediagrams to accommodate and complywith a wide variety of licensing restrictions imposed by agreements withlicensors.

Third, audio and video track synchronization, particularly when they arenot licensed together throughout the entirety of the mediagram, can beproblematic. To solve for this, several of the block diagrams depictedin FIGS. 5A-M insert blank video on the audio-only licensed portions sothat a single video file can be generated and used for personalization.For example, if an excerpt for a music video includes a first portionthat is licensed for audio only—meaning that the mediagram system has anaudio file for the first portion—and a second portion that is licensedaudio and video—meaning that the system has a video file for the secondportion—there may be potential issues with synchronization if the firstand second portions are adjoined to each other with various personalizeduser content. To solve for this, a blank video track can be combinedwith the audio file for the first portion of the excerpt to generate avideo file for the first portion of the excerpt. Then, the video filefor the first portion and the video file for the second portion can beassembled together to ensure proper synchronization between the firstand second portions. After generating this singular video file, thepersonalization can be added and combined to generate the personalizedmediagram.

For example, in FIG. 5A, the mediagram includes a licensed video andaudio excerpt 504 that is combined with personalization sections 502 and506 consisting of instrumental music clip loops and audio chorus clips,respectively. In FIG. 5B, a licensed video and audio excerpt 512 iscombined with personalization sections 508 and 510 with audio files thatare from the different sources. For example, the audio files can consistof in-house created music and/or music from the same source as theexcerpt 512 (e.g., the excerpt 512 and the audio 510 can be from thesame movie).

In FIG. 5C, licensed audio 518 is combined with licensed visuals, whichare then combined with user-designated audio clips used in thepersonalization sections 514, 516. The user-designated audio clips canbe, for example, music that is uploaded by the user or selected using amediagram template for the licensed audio 518. The video excerpts canconsist of label approved visuals 518 (e.g., album art, concertphotos/pictures, concert video, photo shoots, etc.). Label approvedvisuals 518 can be used, for example, when there is not an availablemusic video, or when label approved visuals are more appropriate thanthe music video.

In FIG. 5D, personalization section 520 consists of complete audiosection, and personalization section 522 consists of looping segments ofa song, e.g., associated with licensed video audio excerpt 524.

In FIG. 5E, there is a solid piece of licensed or user-generated audio526 over personalization 528. In this example, there is no videoexcerpt, and voice audio can take precedent over the audio file.

In FIG. 5F, a licensed video audio excerpt 530 is preceded bypersonalizations 532 and 534, each having a duration of atemplate-specified or template-aided length of time 536, and each beinggenerated using a template filter 538 that can automatically fitpersonalization content around a video excerpt. For example, thepersonalized pictures and text can be a set phrase from the song, butdepending on length of the phrase, this time can vary. For instance, ifthe phrase is three seconds long the app will use three secondincrements to determine the length of each personalized option. This canseamlessly lead into the excerpt, and can create a standard for thetiming of the personalization sections. For instance, a picture can bethree seconds and a text box can be three seconds long, which can causea video to need to fit into a divisible of three seconds. Forpersonalized videos, the app can put the “extra” seconds at thebeginning of the video with a template filler, which can solve atechnological issue with fitting the personalization section. The loopcan be either a continuous video or singular audio loops. For singleaudio loops, they can be programmatically assembled with personalizedmedia content on the fly. For the video loop, the audio can be, forexample, a looping phrase of the song and the visual can be a blackscreen. The loops can be pre-combined into a single file on thebeginning and end of the excerpt, with a black screen over whichpersonalization can be added. The program can be designed to start atthe beginning of a loop, with a maximum number of permitted loops for amediagram. Technological aspects of such a configuration include, atleast, a portion of the audio from the licensed music video being chosenfor use as looping audio clip. This clip can be pre-chosen and stored asa supplementary file to the licensed music video. The audio clip can beplayed over the user-supplied content in a loop and then stopped duringthe playback of the music video, for example.

FIG. 5G shows an example mediagram in which the components of the mediacontent being personalized in FIG. 5F (separate loops and video excerptwith audio) are combined into a single, seamless file 540. Creating aseamless file (instead of using the separate component files (loops,video excerpt, video audio) that will be used for personalization), forexample, can resolve technological issues associated with presentingindividual files, such as audio and video files getting out of sync, andcan create a seamless transition that improves the user experience.

FIG. 5H shows an example mediagram that uses a solid stream of audio 542that then transitions into a video excerpt 544. For example, in thisscenario the music and video tracks are licensed for the entiremediagram, even though the video is only presented during the excerpt544 portion. The personalization in this example is placed on the videotrack before or after the excerpt 544. The mediagram system candetermine the start and end times for the beginning and end of thevideo, not the excerpt in the middle, which can provide users will havethe ability to extend and contract the audio of the clip depending onhow much personalization is added before or after the excerpt withouthaving to use fillers or loops. Thus a mediagram can have varyingbeginning and end times within the actual song. Personal images and text(and other personal media) can have a preset duration while personalvideos can vary in length. In this example, the entire music video (andaccompanying audio layer) can be available within the app. The cue pointfor beginning to play the audio layer of the music video would bedetermined programmatically on-the-fly by counting the time-values (e.g.in seconds) associated with each piece of user supplied personalization.The licensed audio layer (no video) portion of the video file can beplayed starting from the calculated cue point. The video portion of thelicensed music video can be masked during that time. At the end of theuser personalization portion of the playback, the audio layer of thelicensed music video can then continue to play, and the video layer canbecome visible for the licensed video playback portion of thepresentation.

FIG. 5I shows an example mediagram in which a user has uploaded a voicerecording for a personalization 546, and an audio clip is created andlicensed by the system for a personalization 548.

FIG. 5J shows an example mediagram in which a video stream of audio 550overlays a variety of personalization options, in which inclusion of avideo excerpt 552 is optionally included.

In FIG. 5K, the audio and video portions of a music video excerpt 556are licensed along with the audio tracks (not video) for thepersonalization sections 554 and 558 before and after the excerpt 556.This scenario can involve the two audio only files and a video filebeing licensed and combined into the mediagram, and can be used, forexample, when sync rights will not permit personalization on top of thevideo portion. Similar to FIG. 5H, personalization before and after theexcerpt 556 can be variable in length.

FIG. 5L depicts an example scenario in which a template for the videoexcerpt guides the user in how to best personalize the mediagram. Inthis example, the template guides the user to designate the mostimpactful, sentimental picture, which the mediagram system canautomatically place immediately after the video excerpt. Alternatively,the mediagram system can permit the user to place personal media contentat various locations and can identify the location where the mostimpactful picture should be placed, which can correspond with the chillmoment.

FIG. 5M depicts an example mediagram 562 that has an introduction 560appended to the start of the mediagram 562 and, in some instances, anend 564 appended to the end of the mediagram 562. The introduction 560can include any of a variety of different combinations of visual contentand audio content, such as the examples 566 a-c (other combinations arealso possible). For example, the introduction 560 can include acombination of preselected introductory visual content and preselectedintroductory audio content 566 a, a combination of personalized visualcontent and preselected introductory audio content 566 b, a combinationof preselected introductory visual content and personalized audiocontent 566 c, and/or other combinations (e.g., combination ofpreselected and personalized visual content, combination of preselectedand personalized audio content). Preselected introductory audio contentcan be, for example, an audio mark (e.g., music or other audio filesthat identify a good or service). Preselected introductory video contentcan be, for example, a visual mark (e.g., logo, animation, name, orother visual content that identifies a good or service). Personalizedvisual content can be, for example, videos and/or images selected by auser (e.g. user-generated photos and/or videos). Personalized audiocontent can be, for example, audio recordings selected by a user (e.g.,user-recorded audio message). The personalized visual and/or audiocontent may extend into and be part of the mediagram 562.

Additionally and/or alternatively, the mediagram 562 can include the end564, which can be similar to the intro 560 in that it can includepreselected and/or personalized audio and/or visual content. Forexample, the end 564 can include a combination of preselected visual andaudio content 568 a, a combination of personalized visual content andpreselected audio content 568 b, a combination of preselected visualcontent and personalized audio content 568 c, and/or other combinations.

FIG. 6 is a conceptual diagram of an example system 600 for generatingpersonalized media content, such as mediagrams. The example system 600can facilitate a user search of a video (e.g., a music video), and canpresent a custom list of videos to the user based on various specifiedsearch parameters. In response to receiving a user selection of one ofthe videos from the custom list of videos, for example, the system 600can integrate various user-provided media items (e.g., audio, video,text, and/or images) with an excerpt of the selected video, based on apersonalization template associated with the selected video.

For example, the system 600 can present user interfaces, such as theuser interface 602, to assist the user in selecting a video for aparticular recipient. A variety of different features can be used toguide selection for the user's self-expression and for the best fitvideo for the recipient. For example, the interface 602 can provide aset of questions to guide the user with selecting the best video toexpress themselves (e.g., provide engagement announcement), notparticularly to a specified person. Libraries can be provided for theuser on each of the personalized content types: text, photos,videos/animations. The style and tone of the library content presentedto a user can be pre-filtered based on prior personalization choicespresented within the app (interests, age, sex, etc.). The specificlibrary content presented can also be based on the category of thechosen music video and/or specific tagged keywords applied to it byadministrators of the app.

In another example, the interface 602 can provide a set of questions toguide the user with picking the best video to tell a specific user aspecified message they want to get across (e.g., provide well wishes tofriend who lost a loved one). In addition to a user being able toprovide explicit search parameters, the system can use data associatedwith the user profile and/or supplied at the time of song selection suchas occasion, age, nature of the relationship to guide song selection andsuggest personalized content. The application can programmaticallyfilter the song list based on tags applied to the song list at thedatabase level. These tags can be similar to the allowed profile choices(age, relationship type, etc.).

In a further example, the interface 602 can prompt a user to answerquestions about the user/or recipient to pick which song to use. Thiscan create a personal list of songs to choose from. Such questions canbe based on, for example, demographics of recipient (age, gender,occasion, favorite genre of music, relationship, message you want to getacross). The answers and contact information (i.e. email address, orother unique identifier) can be stored to create a profile for the userif they sign up with the app. If the recipient responds to the mediagramand signs up, the profile that was saved can populate the recipient'sprofile.

In another example, cached profiles for recipients generated by otherusers can be leveraged in song selection. For example, recipient/userprofiles can be built based on information other users have provided.When finding what song to pick, stored answers and links toaccounts/email addresses can be used to identify songs.

As shown in FIG. 6, the example system 600 can include a personalizedvideo creation system 620. The personalized video creation system 620,for example, can be implemented using one or more computer servers. Insome examples, the computing server(s) can include various forms ofservers, including, but not limited to a network server, a web server,an application server, or a server farm. The computing server(s) may beconfigured to execute application code associated with a variety ofsoftware components (e.g., modules, objects, libraries, services, etc.)and/or hardware components. Two or more software components, forexample, may be implemented on the same computing device, or ondifferent devices, such as devices included in a computer network, apeer-to-peer network, or on a special purpose computer or specialpurpose processor. Operations performed by each of the components may beperformed by a single computing device, or may be distributed tomultiple devices.

The personalized video creation system 620, for example, can providevarious user interfaces (e.g., web interfaces, client/server interfaces,etc.) for presenting information to users through various types of userdevices (e.g., laptop or desktop computers, tablet computers,smartphones, personal digital assistants, or other stationary orportable devices), and for receiving input from the user devices inregard to generating personalized videos. The user devices cancommunicate with the personalized video creation system 620, forexample, over one or more networks, which may include a local areanetwork (LAN), a WiFi network, a mobile telecommunications network, anintranet, the Internet, or any other suitable network or any appropriatecombination thereof.

Operations performed by the example system 600 and the personalizedvideo creation system 620 will be described in further detail below, forexample, with reference to FIGS. 7A and 7B.

FIG. 7A is a flowchart of an example technique 700 for generatingpersonalized videos. The example technique 700 can be performed by anyof a variety of video generating systems, such as the personalized videocreation system 620 (shown in FIG. 6).

User search parameters can optionally be received (702). Referring toFIG. 6, for example, a video search interface 602 can be presented at auser device, the interface including a set of controls (e.g., text inputcontrols, option selection controls, etc.) through which a user canspecify values for one or more parameters to facilitate a search andselection of a video. In the present example, the video search interface602 includes a control for specifying an age, a control for specifying agender, a control for specifying a relationship, a control forspecifying a message to be sent, a control for specifying a preferredgenre of music, a control for specifying a favorite artist, a controlfor specifying a favorite song, and a control for specifying an emotionto be expressed. A user of the personalized video creation system 620may want to send personalized video that incorporates afriendly/romantic/upbeat message, for example. The user can select oneor more appropriate values using one or more corresponding controls inthe video search interface 602, for example, and can submit the selectedvalues to the personalized video creation system 602 as user input 604.

Video options can optionally be displayed (704). In response to the userinput 604 (shown in FIG. 6), for example, the personalized videocreation system 620 can identify one or more videos that match the userselected values of the various search parameters. A corpus of videos(not shown), for example, may be indexed per the search parameters tofacilitate subsequent searches. In response to the user input 604, forexample, a custom list of video options 606 can be presented at the userdevice. In the present example, the custom list of video options 606includes Katy Perry's “Firework,” Outkast's “Hey Ya,” and the Romantics'“What I Like About You.”

A user selection of a video can be received (706). For example, a usercan select one of the videos presented in the custom list of videooptions 606 (shown in FIG. 6) presented at the user device. As anotherexample, a user can submit a video title, a video title and an artistname, or another sort of video identifier. In the present example, theuser selects the Romantics' “What I Like About You,” as indicated byuser selection 608.

A determination is made of whether a preselected excerpt of video isavailable (708). For example, each of the videos included in an indexedcorpus of videos may be associated with a corresponding preselectedexcerpt of the video. A video excerpt, for example, can be a portion ofthe video, and can be of a duration that is less than a duration of thevideo itself. The video excerpt can include one or more impactfulmoments, such as moments for which the video and/or associated music aregenerally recognized, such as a chorus of a song, a popular scene of avideo, or another sort of impactful moment. Video excerpts can bemanually and/or automatically generated.

If the preselected excerpt of video is unavailable, an excerpt of theselected video can be generated (710), as will be discussed in furtherdetail with regard to FIG. 7B.

If the preselected excerpt of video is available, the preselectedexcerpt can be retrieved (712). For example, in response to the userselection 608 (e.g., a selection of the Romantics' “What I Like AboutYou”), a video excerpt 610 (shown in FIG. 6) can be retrieved. The videoexcerpt 610, for example, can be of a duration that is less than that ofthe video, such as fifteen seconds, thirty seconds, a minute, or anothersuitable length of time. In some implementations, a duration of a videoexcerpt may be based at least in part on video and/or musical elementsof the video. For example, beginning and/or end points of a videoexcerpt may occur during scene transitions of a corresponding video,musical transitions (e.g., transitions between a chorus and a verse,transitions to and from solo portions) of the corresponding video, orother appropriate transition points. In general, video excerpts caninclude a continuous audio track from an original video, and can includea segmented video track which includes one or more portions of theoriginal video, and one or more personalization locations for userprovided media. The portion(s) of the original video and thepersonalization location(s) can occur at any position within a videoexcerpt, such as at the beginning, middle, or end of the excerpt. In thepresent example, the video excerpt 610 includes a first personalizationlocation 612 at the beginning of the excerpt, a portion 614 of theoriginal video in the middle of the excerpt, and a secondpersonalization location 616 at the end of the excerpt. The videoexcerpt 610 of the present example also includes a continuous audiotrack 618 from the original video, such that the audio track issynchronized with the portion 614 of the original video.

A prompt can be provided for the user to provide personalized media(714). For example, each video excerpt can be associated with one ormore corresponding personalization templates, which can be retrieved bythe personalized video creation system 620 from a data store ofpersonalization templates 624 (shown in FIG. 6). In general,personalization templates can be used by the personalized video creationsystem 620 to place user provided media in appropriate personalizationlocations of a video excerpt. For example, a personalization templatefor the video excerpt can include locations for user provided text, userprovided video (e.g., including audio), and a user provided image. Inthe present example, the user can be prompted to provide each media itemin accordance with the personalization template, such as through aprompt to “type a hello message for the recipient,” a prompt to “uploada short video telling the recipient what you like about them,” a promptto “upload a funny picture,” and a prompt to “type a goodbye message.”As another example, the user can provide one or more media items of theuser's choice, and the personalized video creation system 620 can matchthe provided media items by media type to a suitable personalizationtemplate for the video excerpt. For example, in response to receiving aseries of images from the user, the personalized video creation system620 can select a suitable personalized template for the video excerptthat is configured to accept media items of the received type (e.g.,images). Templates can suggest stored media content that is appropriateto include with a particular video, such as famous quotes, canned“helper” text/templates for different types of mediagrams, and/orlibraries of artwork, graphics, and pre-generated text.

User-provided media can be automatically placed in one or moredesignated personalization locations in the excerpt of video (716). Forexample, the personalized video creation system 620 can place the userprovided media 622 in designated personalization locations in the videoexcerpt 610 (shown in FIG. 6). In the present example, text provided bythe user (e.g., in response to the prompt to “type a hello message forthe recipient”) is placed in a designated personalization location 622a. A video provided by the user (e.g., in response to the prompt to“upload a short video telling the recipient what you like about them”),for example, is placed in a designated personalization location 622 b.Audio associated with the provided video is integrated with (e.g.,overlaid on) the continuous audio track 618 from the original video, forexample, at a designated personalization location 622 e. The imageprovided by the user (e.g., in response to the prompt to “upload a funnypicture”), for example, is placed in a designated personalizationlocation 622 c. Additional text provided by the user (e.g., in responseto the prompt to “type a goodbye message”), for example, is placed in adesignated personalization location 622 d.

A preview can be provided to the user (718). For example, in response toreceiving the one or more media items from the user, the personalizedvideo creation system 620 (shown in FIG. 6) can generate a preview ofthe video excerpt integrated with the user provided media item(s). Thepreview can be provided to the user at the user's device, and the usercan be given an option to modify and resubmit the media items.

A personalized video (e.g., mediagram) can be finalized (720). Forexample, the personalized video creation system 620 (shown in FIG. 6)can finalize the personalized video, including generating a file of aselected type (e.g., .AVI, .FLV, .GIF, .MOV, .WMV, .MP4, etc.). Afterfinalizing the personalized video, for example, the file can be sent toone or more selected recipients and/or posted to one or more socialmedia platforms.

For example, a proprietary file format for the video 610 can be used forpersonalizing the video to allow for the smooth and accurate playback ofthe various elements involved (photos, looping audio, licenses video,text, etc.) a single video file should be constructed in a specialfashion. The video file can include three segments. First, a leadsegment (612) can include a blank/black video track and looping audiorepeated in the audio track. This lead section can be standardized to beat least of a certain length capable of playing during the time thepersonalized content would be displayed. Second, the middle segment(614) of the proprietary file can include the licensed music video andthe accompanying licensed track. Third, the last segment (616) of theproprietary file format can again include blank/black video as loopingaudio played on the audio track.

The use of this file configuration can be to programmatically determinethe length of time the personalized content needed to be displayed andcue the video file (programmatically) at the proper point within theblank video and looping audio segment, over which, the photos and textis displayed. When the personalized content display period has completedthe looping audio track can seamlessly transition (within the underlyingvideo file format) to play the licensed music and audio segment of thefile. Finally, a smooth transition can occur to the looping audio onceagain (located in the final segment of the proprietary file) asadditional personalized content can be displayed over the blank videoportion of the file. Note that the looping audio could alternately bereplaced by a seamless audio track which transitions into and out of thelicense video segment. In this way the format is flexible to allow forboth kinds of audio content to play during the personalizedcontent—looping or seamless. A benefit of this approach is to reduce thecomplexity of synchronizing content programmatically and push theorganization work into the content editing and preparation process, andalso circumventing potential issues that sequencing the audio togetheras individual files could introduce such as small gaps or glitches inthe audio playback moving from one file to the next.

FIG. 7B is a flowchart of an example technique 750 for generating anexcerpt of a selected video. The example technique 700 can be performedby any of a variety of video generating systems, such as thepersonalized video creation system 620 (shown in FIG. 6).

A video can be retrieved (752). For example, the video creation system620 can retrieve a video from a corpus of videos. Retrieving the video,for example, can be performed in response to a user selection of avideo, when a corresponding preselected excerpt of the video isunavailable. As another example, one or more videos can be selected forautomatic generation of corresponding excerpts, and the selectedvideo(s) can be retrieved for further processing.

The video can be automatically analyzed to identify one or moreemotionally impactful moments (754). For example, the video creationsystem 620 can automatically analyze the retrieved video to identifyemotionally impactful moment(s) in the video, such as portions of thevideo and/or associated music which are generally recognized as causingan emotional impact.

In some implementations, automatically analyzing the retrieved video caninclude performing an automatic analysis of the video content and/orassociated audio content. For example, video analysis of the videocontent can be performed to identify portions of the video which includea close up of a performer in the video of a particular duration (e.g.,several seconds), which may be associated with an emotionally impactfulmoment. As another example, audio analysis of audio content associatedwith the video can be performed to identify portions of the video whichinclude various musical transitions (e.g., significant volume levelchanges, key changes, transitions between solo instrumentation andsinging, etc.), which may be associated with an emotionally impactfulmoment. As another example, text analysis of time indexed lyricsassociated with the video can be performed to identify portions of thevideo which include lyrics that correspond with a song title associatedwith the video, a particular topic (e.g., love, happiness, etc.), oranother sort of lyric that may indicate an emotionally impactful moment.

In some implementations, automatically analyzing the retrieved video caninclude performing an automatic analysis of user interaction dataassociated with the retrieved video. For example, user interaction datamay include video play data for the retrieved video from various users.Identifying emotionally impactful moments, for example, can includeidentifying portions of the video which are frequently replayed byusers. As another example, a video presentation platform may provideusers with an option for generating video clips, and identifyingemotionally impactful moments can include identifying portions of thevideo which have frequently been included in user generated video clips.As another example, a video presentation platform may provide users withan option for indicating a point in time in the video to commenceplayback, and identifying emotionally impactful moments can includeidentifying the point in time that has frequently been selected.

For example, emotionally impactful moments can be identified by usingone or more of the following features:

referencing song portion/hook start and stop times in third-party onlinelibrary,truncating song based on lyric analysis and corresponding lyrictimestamps in the song,waveform analysis to identify optimal ‘chill’ moment based on dramaticwaveform changes, such as changes in verse/chorus, pitch, tempo,loudness, verse/chorus transitions based on changes inamplitude/frequency, determine hook from repeated waveform sections,and/or determine instrumentation change moment, e.g. small ensemble tofull orchestra,Tagging the hook, chorus, intro, outro, and other song features,Analyzing beats-per-minute (BPM) data to determine, for example, whethera chill moment exists within the song (e.g., ballads are slower andoften do not include chill moments, whereas dance-able tunes are fasterand more often include chill moments),Identifying the first iteration of optimal excerpt, e.g. the firstverse/chorus transition, as the primary ‘chill’ moment sets the stagefor the rest of the song,Assigning weights to selection criteria, such as verse/chorustransitions being given more weight than repetitious outros, additionalweights being given to input reflecting user preferences, profile,relationship to recipient, and additional weights being assigned tomusic visuals referencing iconic sections, andModeling the classification criteria around human utterances, such aslower frequencies being associated with more somber mood, whereas higherfrequencies indicate alertness, excitement, and slower tempos triggerreflection, faster tempos inspire motion (entrainment)—a melody's pitchgenerally rises and then falls, which can allow for following the‘melodic arc’ to identify phrases.

Chill moments within media content can be based on a variety of factors,such as mode, mood, tone, pitch, tempo, and/or volume. Many of thesefactors couple together and are used in tandem, and a combination ofthese factors (2 or more factors) can provide chill moments. To fullycapture a chill moment in a way that emotionally impactful, an excerptscan be shorter in length, such as from 10 seconds up to over a minutefor the music video portion of the song.

As discussed above, chill moments, also known as goose bumps and shiversdown the spine, are physiological reactions to visual art, speech,beauty, physical contact and music. The autonomic responses can includeincreased heart rate, increased respiration, forearm muscle activity,increased skin conductance and forearm pilo-erection (hair-raising).Chills induced by music are evoked by temporal phenomena, such asexpectations, delay, tension, resolution, prediction, surprise andanticipation. Chills are evidence of the human brain's ability toextract specific kinds of emotional meaning from music.

Neural mechanisms of chills involve increased blood flow in the brainregions responsible for movement planning and reward, specifically thenucleus accumbens, left ventral striatum, dorsomedial midbrain, insula,thalamus, anterior cingulate, supplementary motor area and bilateralcerebellum. Decreased blood flow in the brain during chill moments hasbeen observed in areas known to process emotions and visual stimuli,namely the amygdala, left hippocampus and posterior cortex.

Chill moments are most often generated by stark musical contrasts, e.g.dramatic changes in mode (minor to major), loudness (soft to loud),tempo (slow to fast), mood (sad to happy), tone (dull to bright) andpitch (low to high). Lyrical passages can trigger chill moments;however, the effect is secondary to the musical effect.

People are most likely to experience chills when listening tunes withwhich they are familiar and have learned to appreciate, comparing whatthey are hearing with their recalled musical model. The music buildstension up to the chill moment (the build-up is longer for romantic,mood-changing chills), at which point there is a resolving release and aconcomitant emotional/physiological response.

Chill moment identification and use in mediagrams to provide emotionallyimpactful messages to recipients can be specifically accomplished usingthe techniques and systems described throughout this document.

Referring back to FIG. 7B, a starting point for a video excerpt can beidentified, based at least on part on a target emotionally impactfulmoment (756). For example, the target emotionally impactful moment canbe identified based at least in part using automatic analysis. Thestarting point for the video excerpt can be designated as occurring atthe beginning of the target emotionally impactful moment, or can bedesignated as occurring at a point in time before the beginning of themoment. In some implementations, the point in time before the beginningof the moment can be a predetermined amount of time (e.g., 15 seconds,30 seconds, etc.). In some implementations, the point of time before thebeginning of the moment can be automatically selected based at least inpart on video and/or musical elements of the video. For example, thebeginning point of a video excerpt may occur during a scene transition,a musical transition, or at another suitable transition point.

An ending point for the video excerpt can be identified, based at leaston part on the target emotionally impactful moment (758). The endingpoint for the video excerpt can be designated as occurring at the end ofthe target emotionally impactful moment, or can be designated asoccurring at a point in time after the ending of the moment. In someimplementations, the point in time after the ending of the moment can bea predetermined amount of time (e.g., 15 seconds, 30 seconds, etc.). Insome implementations, the point of time after the ending of the momentcan be automatically selected based at least in part on video and/ormusical elements of the video. For example, the ending point of a videoexcerpt may occur during a scene transition, a musical transition, or atanother suitable transition point.

For example, designating the starting and ending point can includeautomatically identifying natural and seamless entrances and exits ofthe excerpt. In particular, the automatic identification can avoidjarring, altering pitch, dead air, off beat, unnatural entrances andexits to the excerpt. Additionally, the automatic identification canestablish complete messages and sentiments, thoughts, ideas, phrases,etc. for the excerpt (not truncating messages).

One or more portions of the video excerpt can be designated forpersonalization (760). For example, the target emotionally impactfulmoment can be associated with a duration (e.g., based on automaticanalysis of the retrieved video and/or user interaction data associatedwith the video), and portions of the video excerpt that occur outside ofthe duration of the moment can be designated for personalization.

The video excerpt can be finalized for personalization (762). Forexample, portions of the video excerpt that are designated forpersonalization can be removed, transition effects can be applied suchthat video and/or audio appropriately fades in and out, and othersuitable finalization techniques can be applied to the video excerpt.After finalizing the video excerpt, for example, it can be added to acorpus of preselected video excerpts.

The techniques 700 and 750 can allow for users to contribute to theselection and incorporation of videos and other media content into thecatalogue of media content that is offered for personalization. Forexample, record searches (song, artist, genre, occasion, mood, versionof the song) can be used to identify media content that is of interestto users. Aggregation of the record searches can be used as implicitrequests for media content to be added to the library. In anotherexample, users can provide explicit requests for particular songs byfilling out fields (song, artist, etc.) that identify, for example, thelyrics of the portion of the song they are interested in, and/ornotation of which verse they are requesting. Users submitting requestscan be notified when songs are added.

In another example, streaming media services (e.g., SPOTIFY, PANDORA)can be incorporated into the system to permit near limitless selectionfrom existing databases and instantaneous access. Such services maypresent various disadvantages, such as copyright issues, song versionvariation, spoofed titles, and improper truncation that would have tohappen on the fly and decrease mediagram quality.

In another example, a user's device song library could be used as asource of new media content for incorporation into mediagrams and,possibly, into the system library.

FIG. 8A is a conceptual diagram of an example social media platform 800for providing improved and more meaningful social interactions amongusers. The platform 800 can include a variety of features that provide avariety of benefits over conventional social platforms. In particular,features of the platform 800 aim to build better relationships amongusers by promoting sincere social interactions (no shallowinteractions), to put emotion and meaning back into social media, toprovide both sender and receiver with insights (whether realized ornot), to offer private person-person communication (as opposed tocommunication in front of a broader audience), to provide a fun/playfultone that makes relationships easier to maintain and more rewarding, toassist users in conveying and sending sentiments more accurate to actualfeelings (versus free form text), and to provide social media that canbe useful and uplifting to all users, including introverts.

The platform 800 improves upon social platforms in a number of ways. Forexample, the platform 800 uses gamification to create scarcity withinthe platform, such as scarcity with the number of mediagrams that can begenerated and distributed on the platform 800, and scarcity with regardto the frequency of interactions among users. A relationship conciergecan also employ scarcity in providing prompts at a deliberate pace(e.g., one prompt per day), which can cause users to wait for the chanceto use “high value” prompts (i.e., scarcity can cause users to use thesehigh value prompts less frequently). Scarcity on the platform 800 canalso mimic real-world interactions (e.g., receivingcards/gifts/sentiments) among users that are less frequent than onsocial platforms and, in general, more meaningful. Scarcity on theplatform 800 can also reduce burn out among users and can promoteregular schedule of usages.

The platform 800 can also draw on game theory to improve socialinteractions and relationships. For example, subtle visual and audiocues (e.g., message being concealed and unwrapped like a gift) can beused when viewing/responding to delivered prompts to enhance theemotional state of the user when viewing/receiving the delivered items.Rewards can be used to increase behavior, such as rewards forimprovements in a relationship (e.g., more frequent interactions, moremeaningful interactions) and/or establishing new relationships. Suchrewards can include, for example, points, ratings, icons, symbols,branding methods, emojis and/or other features to represent relationshipstates.

The platform 800 can also incorporate a relationship concierge that canhelp users improve variety, depth and frequency of communication withinrelationships. Such a relationship concierge can use artificialintelligence (AI) algorithms and systems to predict interests, supplycontent and guide the user towards more meaningful relationships. Forexample, the relationship concierge can understand who the peopleinvolved in a relationship by creating a smart wall that prompts theusers to interact with each other on the wall in particular ways toimprove and maintain their relationship. The relationship concierge canbe fed information about users (e.g., interests, demographicinformation) and their relationship (e.g., common interests, type ofrelationship), and can churn on that information with its AI techniquesto determine and provide, for example, to insert prompts directly intothe user's shared private wall to facilitate continued and improvedcommunication. To avoid annoying some users and to allow for variedinterest in the relationship concierge, its involvement can be adaptedto match user preference (e.g., increase or restrict involvement inrelationship). Such interest in the relationship concierge can beexplicit (e.g., user-designated concierge settings) and/or implicit(e.g., user liking or disliking certain prompts from the concierge).

The platform 800 can decrease the anxiety associated with social networkinteractions being in front of a broader audience, which can causeusers' interactions to be more guarded and less authentic, through theuse of private walls that one-on-one between users. With private walls,only the participants in the wall are able to view/contribute to theconversation. Prompts from the relationship concierge can be presentedon private walls. For example, prompts can include questions (e.g.,individual questions, instructions, ideas for topics, joint questionsthat are asked to both users with the answers only being presented ifboth users answer), drawing pictures (e.g., draw pictures and send toeach other), games (e.g., creating a story line by line, hang man, 20questions), challenges (e.g., can take a snap of yourself doingsomething fun/unique, user-designated challenges), articles, pictures(e.g., creating memes and comment on pictures, Rorschach test, photohunt), other creative options (make picture, memes, art, jokes), and/orother options. Private walls can use extrinsic stimulation (e.g., usingcolors, movement and sound to keep users attention) and intrinsicstimulation (e.g., creating an environment that fosters an intimateconnection) to engage users. Such private walls can, for example, createenvironments that fosters communication among both extroverted andintroverted individuals who are looking for social media that is moreprotective and thoughtful than traditional social media, that includesmore intimate communication using media, and that offers protection,reassurance, and control over messages (e.g., knowledge of who sees themessages, who can see the messages, time-limited duration).

As noted in the previous paragraph, the platform 800 can use temporalaspects to reduce anxiety and uncertainty. In one example, messages canhave a lifespan and will be inaccessible to users/deleted from theserver once they expire. In another example, users can only viewmessages for a limited number of viewings, a limited number of times,can only be viewed after a specified period, etc. Such features (timelimits, view limits) can be controlled and designated by the user.Similarly, the platform 800 can permit users to create messages that aresent at a predetermined time. (e.g., sent next Thursday at 10 am) and/orafter an event has occurred (e.g., user returns from vacation).

Additionally, the platform 800 can provide and security measures inplace to provide assurances and protection to user privacy, such asprivate walls being restricted to the participants and/or controlsrestricting and notifications regarding screenshots taken of content.For example, the platform 800 and mobile apps running on client devicescan prohibit forwarding messages outside of the app, alteration of ashared wall to being accessible to the participants, the taking ofscreenshots (recipient is notified if screenshot is attempted), thedevice's ability to copy and paste text, and/or images, downloadingpictures and messages, and/or forwarding content to other users.

In addition to private one-on-one walls, the platform 800 can providegroup walls that, similarly, are restricted to only the participantswithin the group. Group walls can be shared by more than two members andcan creates a venue to share thoughts, ideas, commentary on topics, aswell as a place to share pictures, videos, and other media content withspecific people. Each user who is part of a group can view allcomments/postings in the group. Each group can have an organizer whocontrols the group through group membership, topics, lifespan,moderation, and/or other group parameters. Members of a group cancontribute to conversations, but are not permitted to control groupparameters. The organizer can be identified to the group. As withprivate one-on-one walls, group walls can also have relationshipconcierges that help supply and insert content into the group, such astopics of common interest (either explicitly identified by the group orimplicitly determined from user preferences). For example, therelationship concierge can prompt a group wall with different mediatypes, such as pictures, questions, games, current news articles, memes,“good news” stories (e.g., stories that are relevant and positive. Aimedat creating thought provoking and inspirational dialogue), pop culturequestions, themes, and/or other features. Also similar to privateone-on-one walls, group walls can include temporally limited content aswell as having a time-limited existence. For example, the organizerand/or system can set a lifespan for the group, which can be noted tothe group, after which the group will automatically dissolve and all ofthe content from the group wall will be deleted. Group walls can fosteran environment for “self-regulated” discussion/sharing groups, which canpermit the organizer and/or group members to remove users from thegroup, either through organizer admin approval, a vote of the users,and/or other features. Content within the group wall can beautomatically analyzed, flagged, and deleted if deemed inappropriate(e.g., trolling, hate speech).

Referring back to FIG. 8A, the system 800 includes a social and mediaplatform 802 that provides a social platform, as described in thepreceding paragraphs, as well as a media personalization platform, asdescribed above with regard to FIGS. 1-7. The platform 802 operatesusing a variety of different data sources, including video excerpts 804,personalized videos 806, personalization templates 808, user profiles810, relationship profiles 812, and social data 814. The user profiles810 can include user information (e.g., demographics, interests,location) and can model user behavior. The relationship profiles 812 caninclude relationship information (e.g., users involved in relationship,type of relationship, duration of relationship) and can model therelationship (e.g., state of the relationship). The social data 814 caninclude the data on social interactions between users (e.g., messages,posts, prompts, responses to prompts, content views) and other data forthe social platform 802.

In the depicted example, user A (associated with computing device 816)and user B (associated with computing device 846) have a private wallfor their relationship on the platform 802. A relationship conciergerunning on the platform 802 can periodically determine whether and whena prompts should be provided to one or more of the users A and B to helpfacilitate their relationship. As part of the relationship conciergeprocess on the platform 802, the relationship between the users A and Bcan be analyzed (step A, 826). Such analysis can include evaluation of avariety of factors and data, including the profiles for the users A andB, the profile for the relationship between users A and B, analysis ofhistorical interactions between the users A and B (e.g., determining arating for the relationship), and/or other factors. Based on theanalysis, the platform 802 determines that a prompt should be providedto user A (step B, 828).

In response to that determination, the prompt is provided to the device816 for user A (step C, 830) and is presented (822) on the private wall818 in sequential order with other interactions 820. The private wall818 includes an interface 824 for the user to respond, user input isreceived and provided to the platform (step D, 831-832).

The platform 802 can receive and store the response (step E, 834) andcan determine a minimum time delay for user B to respond (step F, 836).The time delay can vary depending on a variety of factors, such as thestate of the relationship, a current trend of the relationship (e.g.,becoming closer, becoming more distant), and/or other factors. Once thetime delay has been determined, the response 840 and the time delay 842can be transmitted (step G, 838).

The device 846 for user B can receive and present the response in theprivate wall, which includes the earlier message 820, the relationshipconcierge prompt 848, and the response 850. Based on the delayinstructions 842, the device 846 can automatically restrict input beingprovided (via the input interface 852) to reply to the response 850until after the delay has expired.

The platform 800 and the devices 816 and 846 can repeatedly performthese operations A-I in the back and forth communication between theuser A-B, which is configured in such a way by the platform 800 so as toenhance the quality of the social interactions.

FIG. 8B is a conceptual diagram of another example social media platform860 for providing improved and more meaningful social interactions amongusers. The platform 860 can include a variety of features that provide avariety of benefits over conventional social platforms, and can besimilar to the platform 800. In particular, features of the platform 800aim to build better relationships among users by promoting sinceresocial interactions (no shallow interactions), to put emotion andmeaning back into social media, to provide both sender and receiver withinsights (whether realized or not), to offer private person-personcommunication (as opposed to communication in front of a broaderaudience), to provide a fun/playful tone that makes relationships easierto maintain and more rewarding, to assist users in conveying and sendingsentiments more accurate to actual feelings (versus free form text), andto provide social media that can be useful and uplifting to all users,including introverts.

For example, the platform 860 includes a text messaging system 868 thatpermits free form, direct messaging between users 862-864 (one-to-onemessaging and group messaging). The users 862-864 can generate anddistribute the content between each other using the text messagingsystem 868, such as through entering text (e.g., SMS message), providingmultimedia content (e.g., mediagram, videos, photos, MMS message),and/or other content. The platform 860 can also include algorithms andan artificial intelligence (AI) system 870 that can use AI andalgorithmic logic to dynamically infuse the text-messaging interface 868between two users 862-864 with special selected and curated content 866that can help facilitate more meaningful interactions and relationships.For example, the algorithms and AI system 870 can model a relationshipbetween the two users 862-864 and use that model to select particularcontent from the curated content 866, and can present that selectedcontent at particular points in time to facilitate the relationshipbetween the users 862-864. The selected content can be injected into thetext messaging system 868 in any of a variety of ways, such as bypresenting the content to one of the users 862-864 to prompt that user'sinteraction with the other user, presenting the content to both of theusers 862-864 to facilitate interactions among the users, and/or othermechanisms. The algorithms and AI system 870 can continually improveupon and refine its relationship model for the users 862-864 based oninteractions between the users on the text messaging system 868 andtheir response to content injected into the text messaging system 868 bythe algorithms and AI system 870, which can allow the platform todeliver content and experiences that enrich the relationship.

As discussed above, the content that is distributed by the users 862-864and/or selected from the curate content 866 can be any of a varietycontent, including content excerpts (e.g., 2-6 second clips frommediagrams, 2-6 second clips from videos and/or music). Such contentexcerpts can be extracted from any of a variety of different contentsources, such as music videos, television shows, live television shows(award shows), media (news), movies, sound bites from various media,mediagrams, and/or other content. Content excerpts can be sent as quickself-contained messages and/or in conjunction with other messages, forexample, to enhance the overall impact of the messages. Similar to amediagram, content excerpts can be sent without attaching the underlyingcontent from which the excerpt is extracted and/or without associateduser messages. In some instances, words and lyrics of the contentexcerpts can be included and/or transmitted with the content excerpts.

FIG. 9A is an example system 900 for providing an improved social mediaplatform with more meaningful social interactions among users. Theexample system 900 includes a social and media platform 902 (similar tothe social and media platform 802), the databases 804-814 describedabove with regard to FIG. 8A, and user computing device 924. Theplatform 902 can include one or more computer servers, such as cloudcomputing systems. The platform 902 includes a media personalizationsystem 904 with a media analyzer 908 that analyzes media content toidentify excerpts for personalization, a personalization assistant 910that guides users through the personalization processes described abovewith regard to FIGS. 1-7, and a media finalizer 912 that assemblespersonalized media content (e.g., mediagrams).

The platform 902 also includes a social media system 906, which includesa relationship analyzer 914 to determine the state and rating forrelationships (see example technique 2000 in FIG. 20, which can beperformed by the relationship analyzer 914), a relationship concierge916 to prompt and facilitate meaningful social interactions among users(see FIGS. 11A-F, 12A-H, 15A-D, and 18A-B, and corresponding descriptionbelow), a group relationship manager 918 that regulates group walls andgroup interactions (see FIGS. 16 and 21A-B, and correspondingdescription below), interactive games 920 that are used on private andgroup walls to allow users to play games alone or together on theplatform (see FIG. 11B and corresponding description of option 1132),and an interactive wellness app manager 922 that provides features forusers to self-rate their wellness state and to allow for wellnessrating-related interactions among users (see FIGS. 17A-H andcorresponding description).

The relationship concierge 916 evaluates whether and how to prompt usersusing a variety of relationship data representation and analysistechniques. For example, various values attached to questions (or otherprompting in the database 814) can be stored, updated, and evaluated ina hierarchy to determine timing and nature of prompting delivered to theuser. Promptings can have various database values, such as one or moreof the following:

-   -   a “seriousness rating” value in the database (e.g. 1-100) that        indicates how light or heavy the nature of the subject is or        question is,    -   a “nature of relationship” value that indicates the type of        relationship between two individuals (e.g., father, brother,        coworker, etc.),    -   a “topic” value that indicates the category of content (e.g.,        cars, politics, personal history, food, etc.),    -   a “current relevance” value that indicates whether the subject        is uniquely topical to current news or events,    -   a sine wave like pattern that is applied around the seriousness        range that is determined to be appropriate as a current starting        point for a prompt (e.g., if 40 is the target number, prompting        may be delivered in a pattern such as “30, 35, 40, 45, 40, 35,        30” etc.),    -   topics can be favored or rejected (e.g. no politics) and stored        as various values depending on user responses (e.g., stored as        “never user” if rejected strongly, stored as a modifier value if        favored),    -   the center point of the sine wave pattern of delivery can be        “nudged” little by little by the user for responding to “More        Serious” or “Less Serious” preference promptings,    -   frequency of promptings can default to a standard value (e.g., 1        prompting/day), but promptings may be delivered at different        times throughout the day using, for example, a sine wave (or        random) pattern centered on a given starting point (e.g.,        provide appearance of unpredictability),    -   add additional variation by introducing randomization that may        lead to skipping entire day(s) and/or possible delivering more        than one prompting in a day,    -   Record response times (time of day) in database for user, which        can be analyzed for most activity to form a favored usage time        to be applied to questions or contacts that a user is “stuck” on        (i.e., user's relationship with other user is not progressing or        has stalled out), and    -   general “level” value that can be applied to each prompting in        the database, and used to tally an overall ad cumulative “score”        for an individual user if they choose to answer a given        question. The cumulative score can be used to set milestones        crossed that are either hidden from the user, or exposed to the        user (i.e. displaying a tiered “level” you have achieved in the        app with a particular friend).

The user computing devices 924 can each include a mobile app, nativeapp, and/or web application 926 running on the devices 924 that provideinterfaces and features for the social platform (e.g. private walls,group walls, personalized media content) directly to users. Theapplication 926 includes a media personalizer client 928 that presentsmedia personalization features in a user interface and that communicateswith the platform 902 to create media personalization. The application926 also includes a media player (e.g., generic media player,special/secure media player) and a social media client 932 thatimplements client-side of the features for the platforms 802 and 902described above (e.g., implements time delayed responses, private wallsharing prohibitions). The application 926 also includes an interactivegames client 934 to provide interfaces for interactive games and aninteractive wellness client 936 to provide an interface for a wellnessrating and interaction service provided by the social media system 906.

FIG. 9B is diagram of an example system 940 for providing an improvedsocial media platform with more meaningful social interactions amongusers. The example system 940 is similar to the system 900 and includescomponents that can be used to implement the social and media platform902 (similar to the social and media platform 802) and the databases804-814 described above with regard to FIG. 8A. For example, the system940 includes a concept model 942, a data model 946, and a content model950 that are used to model processes, user relationships, content, andother details that are used to provide the improved social mediaplatform.

The concept model 942 can include data, rules, coding, logic,algorithms, computer systems, and/or other features to implement andprovide enhanced social interactions among the users, which can beprovided by, for example, the algorithms and AI system 870 describedabove with regard to FIG. 8B. The concept model 942 can be programmed toimplement various psychological principles that are incorporated intothe underlying rationale of feature and mechanism design and the AI andalgorithmic frameworks that support them. For example, the concept model942 can provide a feature set that addresses the emotive opportunitiesand issues, or affective benefits and costs, associated with remotecommunications, with the end goal being to maximize benefit metrics andminimize or mitigate cost metrics. Benefit metrics can include anemotional expressiveness metric (e.g., metric assessing ease with whichplatform permit users to express emotional states to others and/or toperceive feelings expressed by others), engagement and playfulnessmetric (e.g., metric assessing whether platform facilitatescommunication that is fun and exciting to participants),presence-in-absence metric (e.g., metric assessing whether platformfosters feeling of closeness and/or connectedness to others even thoughseparated by time or space), opportunity for social support metric(e.g., metric assessing platform's ability to facilitate social supportwithout being physically present, such as providing a general sense ofthe other person “being there” for you, reducing negative affect (suchas soothing anxiety), and increasing positive affect (such as feeling“special” or loved)), and/or other metrics. Cost metrics can include afeeling obligated metric (e.g., metric assessing to what extent aplatform creates an unwanted obligation to connect, such as creatingunwanted feelings of obligation or guild to communicate), unmetexpectations metric (e.g., metric assessing platform's propensity tocreate expectations for communication with others that will not be metand, as a result, have a negative impact on participants), threat toprivacy metric (e.g., metric assessing platform's propensity tounexpectedly exposing private information to others, concerns thatothers are eavesdropping on private communication, and concerns thatactions may be invading privacy of others), and/or other metrics.

The concept model 942 includes algorithms 944 a and AI 944 b (e.g.,algorithms and AI 870), interaction mechanisms 944 c (e.g., routinesand/or subsystems to permit and facilitate user interactions),relationship psychology rulebase 944 d (e.g., rules outlining differentrelationship models that can be used to categorize relationships amongusers), and process and data flows 944 e (e.g., processes and data flowsto facilitate improved social interactions among users, includingobtaining implicit relationship feedback from user interactions,refining relationship modeling, and identifying content and timing todeliver the content to the users). Examples of these content selectionsand prompts provided to users are described below with regard to FIGS.11A-F and 12A-H. One the primary goals of the concept model 942 is tohelp users stay connected over the long-term. The periodicity of promptsprovides a cadence (commensurate with the user's wishes for a givenrelationship) of interactions that otherwise might languish due toordinary circumstances of life and the typical dynamics of psychologicaltendencies. Mechanisms which support this goal can include:

-   -   user control over periodicity and level of content delivery,    -   user control over ongoing improvement of content relevance,    -   user anticipation via timing indications of prompting content        visible in the UI, and    -   AI/Algorithms providing relationship-context-aware enhancements.        These and other features can improve social platforms by        facilitating long-term use based on users gaining a perceptible        recognition of some appreciable enrichment to his or her        relationships.

The data model 946 corresponds to the structure and storage of data forthe system 940, including the structure and storage of content, userprofile data, relationship profile data, histories, and/or other data.The data model 946 includes database schemas 948 a (e.g., tabledefinitions, cloud storage database schemas and distribution), datastorage, maintenance, and management procedures 948 b (e.g., datastorage policies, cloud based storage policies), and applicationprogramming interfaces (APIs) 948 c (e.g., APIs to handle server anduser device requests).

The content model 950 corresponds to the content that is delivered tousers on the social platform. The content model includes contentsourcing and creation 952 a (e.g., user-generated content, preselectedcontent, content models that can be adapted to personalize prompts tousers), content psychology rulebase 952 b (e.g., rules definingdifferent types of content and their appropriateness to differentusers), and/or content management procedures 952 c (e.g., processes forcurating content over time). The content model 950 and the data model946 can be used to provide content classifications 958, which can beused to identify relevant content to deliver to users at various pointsin time depending on any of a variety of factors, such as relationshipprofiles, user profiles, and/or other relevant details. The contentclassifications 958 can include content definitions 960 a (e.g.,definitions for different types of content) and content taxonomies 960 b(e.g., hierarchical organization of relationships of different types ofcontent). For example, the classification of content into differenttypes of content can include different configurations of media and data,and can rely on multiple different taxonomies across different datadimensions that function together to more accurately classify contentfor selection and delivery to users. Taxonomies can include, forinstance, a modal taxonomy (e.g., classification of content deliverywhich considers the combination of structural mechanism of delivery andgeneral purpose behind the delivery), topical taxonomy (e.g.,hierarchical classification of the content itself, i.e. topics andsubject matter such arts, sports, history, music, baseball, Mozart, BabeRuth), topical metadata (e.g., non-hierarchical meta-data groupingmethod to allow retrieval and sorting by criteria such as, descriptive(e.g., fun, serious, cultural, academic, controversial), quantitative(e.g., locality, time sensitivity, age appropriateness, complexity levelwithin topic)), and/or others.

Referring to FIG. 9C, which depicts an example system 970 for providingan improved social media platform with more meaningful socialinteractions among users. The example system 970 is similar to thesystems 900 and 940, and includes components that can be used toimplement the social and media platform 902 (similar to the social andmedia platform 802) and the databases 804-814 described above withregard to FIG. 8A. For example, the system 970 includes a content store972 (similar to the curated content 866) from which content is selectedand served to users 990.

In the depicted example, the content store 972 illustrates an examplemodal classification (example of content classifications 958), asdescribed above. This diagram represents a simplified example of modalclassification of content, but other different and/or more complexclassifications are also possible, such as different modes consisting ofdifferent data/media configurations and can each have different handlingin terms of the data modeling and client-side presentation.

The example modal classification includes example data elements 974 a-f.A first element is a shared subjects of interest data element 974 a,which represents content that is specifically relevant to a relationshipbased on a shared interest in a given area of subject matter. This dataelement can be provided at varying levels of specificity, with morespecific categorization aiding in selecting more relevant content forthe users in a relationship. The shared subjects of interest element 974can included fields corresponding to, for example, facts, news,articles, media, and/or other fields.

A second element is a concierge data element 974 b that stores datavalues designed to promote or assist the users' interactions in apragmatic way, which are used by AI logic 976 to select relevant contentfor users. Examples fields for the concierge data element 974 b includereminders (e.g., personal dates such as birthdays, graduations, andanniversaries, holidays such as Christmas, Mother's Day, Father's Day,and Valentine's Day), activity suggestions (e.g., enumerated data fieldincluding designations such as, simple, involved, random, fun, orcontext-specific suggestions of activities for the users), emergent(e.g., prompts resulting from AI/Algorithmic analysis, such asidentifying a keyword from natural language processing such as “dinner”prompting links to local restaurants of shared favorite food types),and/or other data fields.

A third element is a data gather element 974 c that directs in-lineprompts that ask about a user, about other users, about relationships,and/or offer the ability to provide quick feedback about the contentbeing delivered. For example, a normalized data prompts field that ispart of the element 974 c can include prompts that can be stored askey-value pairs, such as “How long have you known each other?”—(Numberof Years); “What is the relationship—Brother, sister, mother, father,uncle, niece, good friend, new friend, co-worker, wife, significantother?”—(Options List); “What interests do you share?”—(Options List);“Would you like more or less prompts with this connection?”—(More/Less);“Did you like the last mediagram message?”—(Y/N).

A fourth element is a programs data element 974 d that stores valuesrepresenting thematic sequences of content and/or prompts. Such a dataelement 974 d can include any of a variety of fields, such asinterpersonal fields that identify current positions along or settingsfor a sequence of prompts that are more aggressively designed to helplearn about the other person, for example a series of prompts onpolitics, or a helpful series of prompts to help in troubledrelationships. Such a field may be configurable by users to have varyinglevels of controversial or difficult questions/content, such as beingconfigured to have a higher likelihood of generating controversial ordifficult questions or content. User interface features can be outputacross multiple different user computing devices so that settingconfigurations are purposefully, voluntarily and mutuallyrequested/agreed upon by the users, as opposed to, for example, beingindependently instantiated by AI or algorithms.

The data element 974 d can additionally include informative fields thatcorrespond to sequences of content and/or prompts that are presented tousers. A variety of different sequences are possible, such as sequencesof content and/or prompts pertaining specific subject matter. Forexample, different sequences of content and/or prompts can pertain tothe history of the French Revolution, basic car maintenance facts, abiography of Steven Spielberg, and/or others. The data element 974 d canalso include entertainment fields that correspond to sequences of mediacontent that are presented to users. For example, content sequences caninclude sequences of short stories, sequences of illustrated series,short comic novels, and/or others.

A fifth element can be an interactive data element 974 e that storesprogrammatic elements (e.g., applications, programs, routines) that canbe run to promote interactions between users at particular points intime. The interactive data elements 974 e can include, for example,drawing programmatic elements (e.g., interactive drawing programs, suchas collaborative drawing programs), game programmatic elements (e.g.,interactive games, such as chess or other strategy games), touch points(e.g., features promoting simple user interactions, such as interactiveimages), entertainment programmatic elements (e.g., videos, music),and/or other programmatic elements.

A sixth element can be a promoted data element 974 f, which can includepromoted content, such as paid advertising content that can be targetedto users based on relationship profiles, user profiles, and/or otherinformation/factors. Promoted data elements 974 f can include, forexample, text, links, images, videos, interactive media elements, and/orother types of content containing one or more promotional messages.

Referring back to FIG. 9B, the concept model 942 and the data model 946can be used to provide profiles 954, such as profiles modelingindividual users (e.g., user profiles) and to model relationshipsbetween multiple users (e.g., relationship profiles for relationshipsbetween two users and/or relationships between groups of users (morethan two users). Profiles 954 can include any of a variety of differenttypes, such as user profiles 956 a, relationship profiles 956 b,relationship histories 956 c, relationship fingerprints 956 d, and/orother profiles. Profiles 954 can be used to identify content that isrelevant for presentation to users based on any of a variety of factors,such as user preferences (as represented by the user profiles 956 a),the user relationships (as represented by the relationship profiles 956b), historical context for user relationships (as represented by therelationship histories 956 c), and relationship fingerprints (asrepresented by the relationship fingerprints 956 d).

Referring back to FIG. 9C, an example of the profiles 956 a-d being usedby an AI logic system 976 to select content from the content store 972for dissemination (980) to user devices 982 is depicted. For example,content can be selected from the content store 972 using relationshipprofiles 956 b, which can include personal user data shared betweenusers as well as all particular information about the relationship.Relationship profiles 956 b can be created by gathering data (984) fromthe user devices 982. For example, users can have full control overviewing, adding to, and editing the content of each of theirrelationship profiles 956 b. Various user-facing mechanisms forgathering and storing relationship-specific information can be used togather data 984 to build relationship profiles 956 b, such as the userdevices 982 presenting a profile user interface for direct user viewingand editing of relationship profiles 956 b (e.g., viewing and editingvarious fields/parameters), in-line social network prompts to obtainquick relationship feedback (e.g., prompts designed to unobtrusivelyallow the user to alter profile settings on-the-fly through quickfeedback, such as through one-click responses), indirect relationshipfeedback from the user devices 982 (e.g., user reaction (or lackthereof) to selected and presented content), and/or other data.Relationship profiles 956 b can include a variety ofrelationship-related data, such as data identifying shared interests,relationship length, relationship type (e.g., brother, sister, friend,co-worker), relationship nature (e.g., serious, light-hearted, romantic,platonic), desired frequency of interaction (e.g., daily, weekly,monthly).

The user computing devices 982 can present user interfaces designed toallow the user to view data and other inputs being used to buildrelationship profiles 956 b. Such user interfaces can, for example,present condensed relationship information on a relationship profiledashboard screen, present graphical visualization based on factors suchas number of shared interests, activity level, number of promptsresponded to, etc., and/or other relationship-related graphicalelements. Examples of user interface features to visualize relationshipsare depicted with regard to FIGS. 13A-C.

The user and relationship profiles 956 b can be generated using the datagathering 984 from the user computing devices 982, through direct andindirect feedback from the users. For example, profile building inputcan be directly gathered through participation by the user as theypopulate user and relationship profiles with information. Userinterfaces will allow users to supply data in a variety of ways, such asinformation supplied about user, information supplied aboutrelationships, and information supplied about other users. In anotherexample, direct data prompts can be provided to users directly askinginformation, such as small portions of information that can, in someinstance, be provide through a “one-click” response, and is easilydismissible by the user in order to be unobtrusive. FIG. 14C is ascreenshot of an example “one-click” feedback interface in which content1472 is presented with selectable graphical elements 1474-1476 that theuser can select with a single click/selection action to provide feedbackrelated to the content 1472. In another example, user answers to contentprompts (e.g., prompts selected a messaging store 978) can be used toconstruct the profiles 956 a-d, such as answers to the questions “whatcolor is your favorite?” presented as clickable grid of colors, or“which historic figure do you admire most?” presented as a selection ofphotos. In another example, usage data indicating how users access anduse the system 970 can be recorded and stored, such as usage dataindicating when users message in a relationship, how often, how quicklythey respond to prompts, where (GPS) do they usually interact with theapp, and/or other usage information. In another example, naturallanguage processing (NLP) can be used to analyze free-form textualresponses from users answering prompts, and can be used to extract andstore key-value pairs and other associated information, such as thefrequency with which key words are used to build a referenceable indexfor reference in AI model inputs.

Referring back to FIG. 9B, user interface features 962 can be providedby combining the concept model 942, the data model 946, and the contentmodel 950. The user interface features can be selected based on, forexample, the profiles 954 and the content classifications 958. Exampleuser interface features are presented in FIGS. 11C-F and 13B-C. FIGS.11A-B show general user interface features, and FIGS. 11C-F presentexample specific user interface features that can be selected forpresentation to users. FIGS. 13B-C present example timing indicators forrelationships.

Referring to FIG. 11C, an example touch point user interface feature1150 is presented. A touch point is a brief interaction prompted by thesystem (identified by “Mora”) that can be more fun than demanding orthoughtful and that can typically have a basic level of interactivity.For instance, the example touch point in this example is a prompt todraw a picture of the other user who is part of the relationship(“Anne”). Other examples of touch points are “tapping one of threeemoji-style faces,” “tapping a photo of Kirk or Picard,” and “drawing asketch (in-line, in-app) of a Mora suggested subject.” Touch points cantarget a presence-in-absence objective.

Referring to FIG. 11D, an example shared experiences (content focus)feature 1160 is presented. A shared experience feature can prompt peoplein a relationship to share a regular but punctuated, periodic andsequenced media experience over an extended period of time. Content canbe based primarily on shared subjects of interest. In the depictedexample, the shared experience feature that is presented (identified by“Mora”) regards Steven Spielberg films, which is an interest shared bythe users. Other examples of shared experience features include “viewingthe biography of a favorite historical figure in a series of bite-sizecontent delivered at a cadence desirable by the user(s),” “playing agame of chess within the interface over the course of weeks or months,”and “sharing thematic sequences of content within the interface is likeslowly watching a TV series together.” Shared experiences can target apresence-in-absence objective and an engagement & playfulness objective.Shared experiences can have a low risk of negative affective costs.

Referring to FIG. 11E, an example relationship concierge feature 1170 ispresented. The relationship concierge can provide reminders,suggestions, assistance, and/or other prompts to assist users withmaintaining and improving their relationships. In the depicted example,a user reminded that it is his/her friends birthday and with asuggestion to send a mediagram (also called a MoraGram), and then theuser acting on that suggestion by sending a mediagram. Other examplerelationship concierge features can include “it's your uncle's birthdaynext week. He likes fishing and camping—how about a related gift?” and“it looks like you're planning dinner—would you like suggestions?”Relationship concierges can target opportunities for social support,engagement, and playfulness objectives.

Referring to FIG. 11F, an example enrichment feature 1180 is presented.An enrichment feature can provide prompts and content which encouragelearning about the other person on a meaningful level. Enrichmentfeatures can have an unassuming approach and can avoids perception ofbeing “clinical.” In the depicted example, the enrichment feature isprompting the user to share a favorite memory with the others user.Enrichment features can target emotional expressiveness objectives.

FIGS. 13B-C present user interfaces with example relationship timingindictors. Timing indicators are a visual display which surfaces theunderlying engagement frequency mechanic, the delivery bufferingmechanic, or both. The variations below describe benefits and risks thatcould result from implementation of these mechanics. FIG. 13B shows anexample user interface with a single timing indicator and FIG. 13C showsan example user interface with dual timing indicators. Other quantitiesof timing indicators are also possible. The timing indictors can presentvisualization for timing related to one or more of the followingrelationship features:

-   -   incoming user messages from system prompts,    -   outgoing user messages from system prompts,    -   timing since last outgoing message,    -   timing since last incoming message,    -   timing since last communication (either incoming or outgoing),        and/or    -   time until system is scheduled to deliver next prompt.

Referring back to FIG. 9B, content programming and sequencing 964combines the concept model 942 and the content model 950. Deliveringusers content that is relevant and specific can be a significantchallenge. If content is too general the perception by the user may bethat they appear to be advertisements. For example, if the user providesa general interest in sports when, unknown to the system, the user has aspecific interest in the Boston Red Sox, attempts to deliver relevantcontent falling under the general “sports” classification can causedisengagement and frustration by the user (e.g., serving content relatedto the NFL, other baseball teams). As a result, the breadth and depth ofthe content model 950 can have a significant impact on the relevance ofcontent that is selected for presentation to users, and ultimately onuser engagement with the system and other users.

The content programming and sequencing 964 can include a variety of dataelements that are being tracked and used to determine when and whatcontent to serve to users, such as engagement frequency, deliverybuffering, ephemerality, privacy of shared information, and/or others.Engagement frequency relates to the level of involvement the system haswith the user and, more specifically, to particular relationships. Auser may set a default value for the desired frequency for prompts andcontent delivered to the user, and can do this individually for eachrelationship. For example, the user may choose to set a high frequency(e.g. daily) for a significant other while setting a very low frequencyfor an old acquaintance (e.g. monthly or quarterly). Frequency settingcan be adjusted through direct and/or indirect user feedback, such asadjusting the timing of these feedback prompts based on analytic data ofactual user behavior. For example, if the user regularly delays aresponse to prompts in a given relationship, a frequency adjustmentprompt would be delivered to the user.

Delivery buffering is a mechanism which purposely delays the sending andreceiving of messages (e.g., prompt responses by users) by a certainamount of time (e.g., hours, days). Delivery buffering is contrary toconventional social media systems which seek to speed up the pace ofuser interactions. Delivery buffering can provide a variety of benefits,such as allowing users the ability to recall messages, as needed, and tobuild anticipation during which both senders and recipients are thinkingabout each other (e.g., incoming message buffering is visuallycommunicated in the UI, such as FIGS. 13B-C).

Ephemerality refers to messages and content sent between users that willbe “removed” after a certain period of time. The window of time thatelapses before content is removed can be controlled by the user(s) perrelationship. This feature can help preserve user privacy. Privacy ofshared information relates to features that purposely limit a user'sability to distribute information shared on the platform will beimplemented. The features include disallowing the ability to copy andpaste content from the app to other applications, and discouraging thecapture of screen contents by use of a devices screen shot feature.Where possible, this device feature would be disabled while using theapp. However, device manufacturers have typically not allowed thedisabling of the screen shot feature. A method of informing the userthat the message-sender is notified of screen shots being taken will beemployed as discouragement.

FIG. 10 is a flow chart 1000 with user interfaces 1002 and 1016 toestablish an initial connection between users on a social mediaplatform. The interfaces 1002 and 1016 can be presented by the socialmedia client 932 on the user computing devices 924, for example. In thedepicted example, an initial social connection is established betweentwo users. As part of this process, the users provide information andanswer questions about each other and about their relationship, whichthe system 906 uses to create and/or improve upon user and relationshipprofiles that are used by the system 906 (e.g., used by the relationshipconcierge 916).

In the first interface 1002, a series of information requests andquestions 1006-1012 are posed to the user for his/her new relationshipwith the user 1004. As indicated by the username for user 1004,usernames can include any of a variety of ASCII characters (includingnon-alphanumeric characters, such as symbols and operators) as well asicons/emojis/graphics (as indicated by the seashell icon). In thedepicted example, the user is prompted to provide the user's desiredprompt frequency for the relationship 1006, the type of relationship1008, common interests among the users 1010, and types of prompts thatthe users are interested in 1012. Responses to this information assistsin initializing the relationship (1014).

In the second interface 1016, the user is again presented with a seriesof information requests and questions 1018-1024. For example, the useris prompted to designate a desired level of concierge involvement in therelationship 1018 (e.g., heavy involvement can cause all interactions topass through the concierge—meaning no freeform exchange outside ofconcierge prompts, minimal involvement can permit many interactionsoutside of the concierge), whether the concierge should prompt one orboth users at a time 1020, the prompt types 1022 that the user isinterested in, and desired minimal delay for users to interact with eachother on the wall 1024. With these parameters selected, the conciergecan be initialized (1026) and the users can begin socially interactionin the platform (1028).

FIGS. 11A-B are screenshots of example user interfaces 1102 and 1114 onan example mobile computing device 1100 for interacting with other usersvia private walls on a social platform.

Referring to FIG. 11A, an example home screen interface 1102 is providesa list 1112 the user's friends on the platform along with relationshipinformation for each of the friends. Each of the friends is identifiedby a username 1106, a relationship status icon 1104 (status of therelationship between the user of the device 1100 and the friend), arelationship rating 1108 (rating of the relationship between the user ofthe device 1100 and the friend), and information on the last interactionbetween the users 1110. More stars for the ratings 1108 indicates astronger relationship, and fewer stars indicates a weaker relationship.Relationship ratings 1108 can be determined based on a variety offactors, such as points for questions and aggregate point summaries overtime. The relationships (as identified by the friends 1106) are sortedin the list 1112 in reverse order so that the relationships most in needof attention by the user are seen at the top of the list 1112. Therelationships in the list 1112 can be selected to navigate to a privatewall for the relationship.

Referring to FIG. 11B, a private wall for a relationship with user 1142is presented in the interface 1114, which includes a variety ofdifferent options 1118-1132 and 1140 for the user to interact with theother user 1142. The private wall 1114 also includes a chronologicalview of recent interactions between the users, which includes unpromptedmessages 1134 and 1136, as well as a prompt 1138 that has been providedto the user. As indicated by the timestamps of the messages 1134-1136and the prompt 1138, over two weeks had elapsed since the users hadinteracted, which is what likely triggered the relationship concierge toprovide the prompt 1138 to continue communication between the users. Theuser can respond to the prompt 1138 through the interface 1140 and/orthrough one or more of the interaction options 1118-1132. The user canalso elect to ignore the prompt 1138 and/or to indicate dislike of theprompt 1138.

The interaction options 1118-1132 include an interactive wellnessfeature 1118 (see FIGS. 17A-H), a prompt feature 1120 to request foranother or different prompt (e.g., step outside of current art ofprompts from the relationship concierge), questions 1122, an interactivedrawing feature 1124, a picture sharing feature 1126, a mediagramcreation and sharing feature 1128, a photo/video sharing feature 1130,and a games feature 1132.

The interface 1114 also includes relationship status information 1144(rating for the relationship with user 1142) and options 1146 to modifysettings for the relationship.

FIGS. 12A-H are screenshots of an example process flow for arelationship concierge facilitating and improving social interactionsamong users via private walls on a social platform.

Referring to FIG. 12A, the user device 1200 for Anne includes theinterface 1204 for a private wall between Anne and David on Monday at3:00, which is when Anne receives a prompt 1206 from the relationshipconcierge. The prompt 1206 is accompanied by a field 1208 through whichAnne can respond to the prompt.

Referring to FIG. 12B, at the same time as Anne receives the prompt1206, the user device 1202 for David does not present any prompts in theinterface 1205, including not presenting the prompt 1206 just given toAnne. This scenario represents an initial state where ho prior prompthistory is visible in David's view.

Referring to FIG. 12C, a few minutes later at 3:05 on Monday, Anneenters and submits an answer 1208 to the prompt 1206 given by therelationship concierge, as indicated by the sent status 1210 for theprompt and answer (1206-1208).

Referring to FIG. 12D, at 3:05 on Monday, David receives Anne's message1212-1216. The question (or directive) given by the relationshipconcierge is visible to David (1214), in addition to the content of herreply (1216). Although not depicted, the prompt response can be an iconthat, once selection, opens up like a gift with animation. Suchanimations features could additionally be used as ways to presentelectronic gifts and/or donations to other users and/or organizations(e.g., charitable donations to disaster victims).

Referring to FIG. 12E, the following day at 1:30, Anne has not receiveda new prompt from the relationship concierge, and any new prompts givento David are not visible in the interface 1204.

Referring to FIG. 12F, on Tuesday at 1:30, David receives a new prompt1218 from the relationship concierge, which includes a field 1220 toprovide a response. The prior history of sent and received prompts isvisible in the interface 1205, but may be removed after a default oruser-set amount of time.

Referring to FIG. 12G, at 1:40 on Tuesday, Anne receives a message1224-1228 from David which displays both the prompt 1226 given to Davidand the content of his reply 1228.

Referring to FIG. 12H, at 1:40 on Tuesday, David has replied 1220 to thelatest prompt 1218 given by the relationship concierge, as indicated bythe sent status 1222.

FIG. 13A is screenshot of an example user interface 1300 on a mobilecomputing device for viewing a user's friends and the correspondinginteraction delays until another relationship concierge prompt isexpected. The user interface 1300 presents a list of friends across anumber of different categories, including a “Msg” column 1302 thatindicates whether the user of the device presenting the interface 1300has a message waiting from one of his/her friends. Such messages caninclude, for example, any type of prompt, a mediagram (personalizedmusic video message), etc. The unopened gift icons 1308 indicate thatthe user has not viewed the waiting message yet. The opened gift icons1310 indicate that the user has already viewed all messages sent fromthe corresponding friend.

The “Name” column 1304 displays the name of the friend(s) with whom you(the user) are having a private conversation. The “Time Until Next”column 1306 indicates an amount of time, which could be eitherapproximate window of time or a precise amount of time, until the nextprompt will arrive for that relationship from the relationshipconcierge. The “Time Until Next” column 1306 could be used to representadditional and/or alternative relationship metrics. For example, the“Time Until Next” column 1306 could indicate timers (bars) representinghow much time has passed since the user last communication with a givencontact. In such a scenario, the longest bar would be shown on top ofthe list to highlight the relationship in greatest need of attention.Color distinctions in the timer bars can indicate an “overdue” statewhere too much time has passed (according to default values or user setvalues).

FIG. 14A is a conceptual diagram of an example personal concierge systemand algorithm 1400 for facilitating and improving user relationships ona social network. The relationship concierge 1402 is a programmed logicthat is designed to interpret and understand the nature of arelationship between two people, tendencies, and behaviors of eachindividual, and to formulate a forward looking program of prompts basedon those factors. The relationship concierge 1402 is largelyalgorithm-based, using historical user data and user inputs to determinethe content of the prompts given to a user. The relationship concierge1402 also incorporates one or more AI techniques and platforms to allowfor decisions to be made that are not pre-programmed into the algorithmor pre-determined. The relationship concierge 1402 can be allowed tomake choices for the users based on emerging patterns of usage and userinput.

The relationship concierge 1402 can use a variety of different datasources 1404-1408 to determine and provide prompts to users. Forexample, the relationship concierge 1402 can use historical userbehavior data 1404, which can include, for example, answers to prompts,how long the user takes to responds, the times of day the user responds,how quickly the respond to certain categories of prompts, how often theydismiss (reject) certain types of prompts, and/or other relevant datarepresenting historical user behavior.

In another example, the relationship concierge 1402 can use useradjustment data 1406 which indicates changes in relationships over time.Users are provided with options to directly supply feedback andinformation on the nature of their relationships with others. Forexample, the user could indicate that the relationship for a contact isintimate/romantic in nature, and further that there has been a recentbreakup in the relationship, and further that they either want tore-kindle the relationship or to ease it into a platonic relationship.In another example, the user could indicate that the contact is an oldfriend that they would simply like to stay in touch with but is notinterested in delving into deep conversations. Other forms of ongoingdirect user input can be indicated via in-line feedback options withinthe ongoing conversation that they liked or disliked a given prompttype, or that they would like the speed up or slow down the rate atwhich prompts are supplied.

In another example, the relationship concierge 1402 can use one or moredefault program of prompts 1408 based on the standard parameters ofinput (described above). In addition to the one or more defaultprograms, specialized sets of prompts can be centered around a themethat can be chosen by the users. These special sets of prompts, ifchosen, can be weighted above the standard parameters. The set ofspecial prompts centered on a theme can have a discrete quantity andstart/end date, not necessarily known to the users. Examples of thespecial themed programs that can be delivered by the RelationshipConcierge can be a series of prompts that have an aim to, for example:reconcile political viewpoint differences, patch up a failingrelationship, deeply explore the memoires and life of an individual(e.g., a grandmother and granddaughter relationship), explore a specificsubject such as philosophy, religious beliefs, and/or lighter subjectssuch as movies, art, music, sports, etc.

FIG. 14B is a diagram of an example system 1450 to vary content that isselected for presentation to users. Content delivery to users can seekto balance user-reported desirability with natural variation to avoidboth extremes of content irrelevance and overly predictable consistency.The example system 1450 can be implemented as part of the exampleplatforms/systems 800, 860, 900, 940, 970, 1000, and/or 1400 describedabove. The system 1450 can effectively utilize a feedback loop toprovide content and, based on user feedback, to refine the selection offuture content that is selected for delivery to users.

Relationship profiles and history (1452) can be used to select content(1454) for presentation to users. The relationship profiles can includedata that describes relevant matching characteristics learned by variousmethods including self-reported data contained in individual profiles,data gathered from algorithms and analytics, and user-reported dataabout the specific relationship. A history of content delivered to therelationship can be stored in order to track, regulate, and plan theflow of content. The content can be, for example, taxonomicallyorganized content is stored in the system which is then queried andretrieved based on relevance to the specific relationship.

Topical interests (1456) can be used to refine the content selection(e.g., pare down a large set of content to a smaller subset of content).Topical interests are qualitative measures of both implicit relevance(e.g. some content is of more general relevance to a married couple thanfriends or co-workers) and explicit relevance (e.g. user-supplied dataindicating a shared interest in baseball or the Boston Red Sox.). Themore specific the domain of interest is defined the greater the value oftopical interest. The classification of content is stored as part of thetopical taxonomy described in a later section.

Intensity (1458) and frequency (1460) parameters can be used to furtherrefine the content selection. Intensity generically refers to where inthe spectrum of casual to intimate (or personal) the nature of thecontent belongs. The intensity of any given piece of content is anattribute applied and stored in the metadata taxonomy. Frequency can be,for example, a quasi-mutually agreed upon value between two usersregulating how often the system will deliver content. For example, ifone user sets the initial desired frequency at daily and the other setsthe desired frequency at weekly then the system may set the startingpoint for the actual delivery frequency at every three days, and is thusa de facto negotiated interaction.

The selected content can then be delivered via one or more modules(1462) (e.g., FIGS. 11C-F). Modules are how content is manifest in theuser interface. Each module type is a specific combination of mediaformatting (e.g. text only, text with image, etc.), interactivitycharacteristics, and categorical purpose (e.g. a reminder, a questionprompt, an element of a thematic sequence (see programs), or a promotedad).

User feedback can be obtained (1464) from the UI and used to furtherrefine the relationship profile and history (1452). For example, ascontent is delivered, the opportunity for users to provide quick“one-click” feedback will be presented. Occasionally buttons that allowusers to tap Less Often/More Often/No Change or More Like This/Less LikeThis will be attached to a piece of system-delivered content. Thisfeedback is used to adjust the relationship profile data accordingly.

FIGS. 15A-D are screenshots of a relationship concierge being applied toother social platforms providing predominantly open communication amongbroad groups of users, such as FACEBOOK, TWITTER, LINKEDIN, and/or othersocial platforms.

Referring to FIG. 15A, an example user interface 1500 on a socialplatform providing predominantly open communication among users isdepicted. The interface 1500 can be a news feed, for example. Theinterface 1500 includes a post field 1502 through which the user cancreate and submit a post for distribution across a broad group of users(e.g., friends, fans, followers, public). The interface 1500 includes arelationship concierge prompt 1504 that is presented to the user. Theprompt 1504 is identified as being from the relationship concierge andthat it is presently only visible to the user (1506). The prompt 1504identifies that it pertains to the User A (user of the interface 1500)not having interacted with User B in over one month and suggests anumber of options (1508). The example options include a first option1510 to like or comment on a recently popular post 1512 of User B. Atleast a portion of the post 1512 is presented in the interface alongwith interactive features 1514 and 1516 through which the user candirectly interact within the feed. These interactions with the post1512, as facilitated by the relationship concierge, will be viewable byand potentially broadcast to a broader audience than just User A andUser B.

A second option 1518 is to publicly post a message on User B's wall.Again, this option includes an interactive feature 1520 to perform theaction from within the feed. Also again, this interaction will beviewable by and potentially broadcast to a broader audience than justUser A and User b.

A third option 1522 is to answer a question for User B regarding UserA's favorite movie over the past year. Again, this option includes aninteractive feature 1524 to perform the action from within the feed.This option, however, provides a select box 1526 through which the UserA can designate whether the answer to this question should be deliveredas a private message (not initially viewable beyond User A and User B,unless forwarded or shared with other users) or posted to a broaderaudience. In this example, the User A enters an answer to the questionin the third option 1522 and does not select the box 1526.

Referring to FIG. 15B, an interface 1528 for User B on the socialplatform presents a post 1532 for the relationship concierge prompt 1504and answer 1524 from User A. The post 1532 includes informationidentifying that the User A answered a question posed by therelationship concierge for User B, the question and answer 1536, andfeatures 1538-1540 through which the User A, the User B, and other userscan interact with the post 1532. The news feed 1528 for the User B alsoincludes a field for the User B to create a post 1530 and a post fromanother user 1542.

Referring to FIG. 15C, the interface 1500 for the User A on the socialplatform is again presented with the prompt 1504 from the relationshipconcierge. However, in this example the user selects the box 1526 todeliver the answer 1524 to the question 1522 privately to User B.

Referring to FIG. 15D, a private messaging interface 1550 (e.g.,FACEBOOK MESSENGER) for the User B on the social platform is presented.The interface 1550 depicts the private message 1552 from the User A aswell as the question and answer 1554 to the prompt 1504 from therelationship concierge. The private message 1552 is presented amongother private and group messages 1556-1566 for the User B on the socialplatform. Unlike the social platforms 802 and 902 described above, theremay be no restriction on the User B transmitting or sharing the message1552 more broadly than just the relationship between the User A and theUser B.

FIG. 16 is a diagram depicting creation and use of a private group wall1600 on a social platform to improve and enhance meaningful socialinteractions. The example group wall 1600 has multiple users 1602 whoare members of the group and who are permitted to contribute to the wall1600. The group organizer is identified at the top of the list with thenotation “organizer.” The organizing user can designate a variety ofparameters for the group, including who is invited/permitted to be amember, permissions for other members to add new members (e.g., friendsof original members are able to be added), time limits on the existenceof the group wall (e.g., 2 month expiration date), roles for differentgroup members to play within the group (e.g., rock band roles—bandmember, groupie, fan), and/or other features.

The group wall can be initiated with conversation starter, which can befacilitated by the relationship concierge. The conversation starter caninclude, for example, pictures, drawings, memes, videos, news stories,questions, etc. If the group organizer needs help finding a topic ofcommon interest, they can use the relationship concierge 1604 to createa custom list 1604 of common interests (which can automatically beidentified from user profile analysis) and can choose a topic 1606 mostof the participants have in common. The selected topic 1606 can be usedto insert initial content 1608 into the wall 1600 that pertains to theselected topic 1606. In the depicted example, the initial content 1608includes news articles relevant to the topic 1606.

FIGS. 17A-H are screenshots of an example user interface 1702 on acomputing device 1700 for users to express and interact with othersregarding their emotional well-being.

Referring to FIG. 17A, the interface 1702 is a visual aid for users tobetter understand their feelings and improve their mental states. Thethree corners of the interface 1702 (a triangle) represent emotionalextremes. The top corner (yellow/white) is selfless compassion, a brightideal to strive for. The left corner stands for passion and the rightcorner represents depression. The center circle 1704 representsnormality, the sphere of daily emotions.

Users can use the interface 1702 to assist in expressing and improvingtheir emotional states, to improve their moods, and gain peace-of-mind.A movable pin icon 1706 can be placed at different positions throughoutthe interface 1702 by the user to represent his/her current emotionalstate. The user can adjust the positioning of the icon 1706 asfrequently or infrequently as he/she wants (e.g., hourly, daily,weekly). Increased frequency of use can assist users in understandingand tracking the change in their emotional state over time, and can helpthem work to improve their moods. For example, the three corners 1710,1714, and 1718 of the interface 1702 represent calm, understanding,enlightened, generous, compassionate (top corner 1710, which can becolored yellow-white); anger, agitated, irritated, passionate (leftcorner 1718, which can be colored red); and sad, depressed, down, bored,dispassionate (right corner 1714, which can be colored blue). The threesides 1708, 1712, and 1716 of the interface 1702 represent optimistic,enthusiastic, upbeat, joyful (left side 1708, which can be coloredorange); friendly, sociable, agreeable, cool (right side 1712, which canbe colored green); and anxious, upset, worried, fearful (bottom 1716,which can be colored purple). The top half of the interface 1702 canrepresent positive, healthy emotions, whereas the bottom half representsnegative, less-healthy emotions

Three different walls on the social platforms 802 and 902 can depict thecurrent mood of the user of the device 1700—a private wall that is onlyaccessible to the user of the device 1700 and the relationship concierge(see FIG. 17B), a shared private wall for a relationship between twousers (see FIG. 17C), and a private group wall for more than two users(see FIG. 17D).

Referring to FIG. 17B, a private wall for the interface 1702 that isonly accessible to the user of the device 1700 and the relationshipconcierge is depicted. The pin 1706 indicates the user's current mood.The pin 1706 can be positioned, for example, by the user with the threesliders 1720-1724—Calm/Anxious, Friendly/Angry, Optimistic/Depressed.The user may also choose to include input from the relationshipconcierge and/or other users.

For example, in the depicted example the user has moved the pin 1706 inresponse to a sad event occurring (e.g., user's pet just died). The userdoes this by moving their Optimistic/Depressed slider 1724 to the rightinto the blue area. This action can update the user's interface in othershared walls and/or group walls for the user, for example, in responseto the user providing permission for it to be shared in that manner.Sharing the interface 1702 can be a way for users to share theiremotional state with others when it may otherwise be difficult toexpress their emotions. In this example, other users who see the user'scurrent state in the interface may be prompted to respond by sendingappropriate mediagrams to the user to help improve his/her mood.

Referring to FIG. 17C, a shared wall is depicted in which the pin 1706for the user of the device 1700 is superimposed on the same interface1702 as another pin 1726 for the other user of the shared wall.

Referring to FIG. 17D, a group wall is depicted in which the pin 1706for the user of the device 1700 is superimposed on the same interface1702 other pins 1726-1730 for the other users who are members of thegroup wall. The current mood of every member in the group can bedisplayed on the interface 1702. Different group walls can addressfeelings about different topics, for example. Members of the group,including the user of the device 1700, may choose to use mediagrams orother interactive/social features to interact with other group toimprove their moods. Users who are able to successfully improve the moodof other users through various actions on the social platform canreceive positive relationship points, which can factor into relationshipratings.

Referring to FIG. 17E, different mood goals can be designated for thecorners of the interface 1702. If, for example, the corners representCompassionate (1732), Impassionate (1736), and Dispassionate (1734), theuser can decide to meditate, reflect on relationships, and/or reach outto other members in order to move their icon (1706) upwards toward theCompassionate (1732) corner.

Referring to FIG. 17F, the interface 1702 can be used to representdifferent strategy vectors 1738-1742. For example, users can imaginealtering their moods along three Strategy Vectors—Engaged/Detached(1740), Caring/Selfish (1738), and Calm/Agitated (1742). Activities forimproving mental states with these strategy vectors can include, forexample, interacting more with other users, helping to solve anotheruser's problems, and self-help (e.g., meditation, exercise, listening tomusic, etc.).

Referring to FIG. 17G, the interface 1702 can be used to representconflict resolution goals 1744-1750, such as Ultimatum (1746), Surrender(1748), Compromise (1750), and Contentment (1744). Users can use theinterface 1702 to resolve conflict by first choosing anapproach—Ultimatum, Surrender, or Compromise—and then adopting astrategy that will lead to Contentment.

Referring to FIG. 17H, the interface 1702 can be used to assist userscoping with grief. For example, a user can follow their progress throughthe 5 (suggested) stages of grief (Disbelief 1752, Anger 1754,Bargaining 1756, Depression 1758, and Acceptance 1760), eventuallyimproving their moods through understanding 1762.

FIGS. 18A-B are flowcharts of example techniques 1800 and 1850 fordetermining and transmitting prompts to specific relationship privatewalls on a social platform.

Referring to FIG. 18A, the example technique 1800 can be for determiningand transmitting prompts to users who share a private wall correspondingto their relationship part of a relationship concierge. The userprofiles for the users sharing the wall and the relationship profilebetween the users can be accessed (1802). Historical interactionsbetween the users via the private wall can be analyzed (1804). The userprofiles, the relationship profile, and/or the historical interactionsbetween the users can be used to determine a current state for therelationship between the users (1806). Such a relationship state can be,for example, a relationship rating or score that is provided to quantifyaspects of a user relationship, such as the quality, closeness, and/orother relationship aspects. The current relationship state can becompared with other relationship states for other relationships that oneor both of the users have (1808). For example, a comparison can be madeto determine whether the current relationship under evaluation isbetter, the same as, or worse than other relationships. The trend of therelationship over time can also be determined by evaluating timesequence relationship states for the users (1810). For example, anassessment can be performed to determine whether the relationship isimproving (i.e., users are becoming closer), staying the same, ordeclining (i.e., users are becoming more distant). Evaluation of currentand trending wellness states that the users have self-reported (e.g.,via the interface 1702) can also be performed (1813). For example, theemotional state of each user may be affecting the relationship betweenthe users and may provide insight into corrective actions via promptsthat could be taken to improve both the user's wellness state and therelationship.

Based on one or more of the relationship factors determined in1802-1813, a determination can be made as to whether to provide a promptto one or both of the users in the relationship (1814). If no promptsare determined, then the technique 1800 can repeat. If prompts aredetermined, then one or both of the users can be identified to receivethe personal concierge prompt based on one or more of the factorsdetermined in 1802-1813 (1816). For example, if one of the users in therelationship has a depressed wellness state and the other user in therelationship has a positive wellness state, the user with the positivewellness state can selected to receive the prompt to interact with thedepressed user (in an attempt to improve the depressed user's wellnessstate). A determination of the type of prompt that should be provided tothe selected user can be made (1818). Extending the previous example, inthe case of a depressed user, the prompt may be for the positive user toprovide something more impactful to the depressed user, like amediagram. Once the user to receive the prompt has been selected and theprompt type has been identified, the prompt can be transmitted (1820).

Referring to FIG. 18B, the example technique 1850 can be for determiningand transmitting prompts to a personal wall for the user and thepersonal concierge alone (no other users permitted on the private wall).The user's profile can be accessed (1852) and can be used to determinewhether any upcoming events exist for the user or the user's friends(1854). At appropriate times, reminders for such upcoming events can beprovided on the personal wall for the user (1856). A determination canbe made as to whether any user-set reminders are upcoming (1858). Atappropriate times, reminders for such user-set reminders can be providedon the personal wall for the user (1860). A determination can be made asto whether any current events or news related to user interests havecome out recently that the user is not aware of (1862). If such currentevents or news does exist, then notifications can be provided by thepersonal concierge on the user's personal wall for those current eventsand/or news.

FIG. 19 is a flowchart of an example technique 1900 for determining andtransmitting delays between interactions on a social platform. For aparticular relationship between two users, the user profiles and therelationship profile for the users can be accessed (1902) and, alongwith historical data for the users and the relationship, can be used todetermine a historical cadence of interactions between the users (1904).The current type of interaction that would be delayed can be identified(1906), a current status of the relationship between the users can bedetermined (1908), and the current relationship trend for the users canbe determined (1910). Based on one or more of the factors determined in1902-1910, a determination can be made as to whether or not a responseto the current interactions between the users should be delayed (1912).For example, if the relationship is currently strong and the currenttype of interaction is a mediagram, then a delay in the response may beappropriate. However, if the relationship is currently weaker and istrending in decline, then either no delay or a minimal delay may beinstructed. Other ways and outcomes for determining whether a delay isappropriate are also possible.

If not delay is needed, then instructions can be provided to permit theuser of the client device to respond without a delay (1915). If adelayed is determined to be needed, then the delay length can bedetermined based on one or more of the factors determined in 1902-1910(1914). For example, if the user relationship is trending upward and theusers typically have a lengthier cadence of interactions, then a longerdelay can be determined. In another example, if the relationship istrending downward, them a shorter delay may be determined. Once thedelay and the delay length has been determined, it instructions forinstituting the delay on a client device can be transmitted (1916).

FIG. 20 is a flowchart of an example technique 2000 for determiningrelationship ratings on a social platform. For a particular relationshipbetween two users, the user profiles and the relationship profile forthe users can be accessed (1902) and the historical interactions betweenthe users can be accessed (1904). Relationship points, which can be atleast one of the metrics by which relationships are rated, can beallocated for each of the interactions (1906).

Allocated points can then be weighted more heavily for interactions thatindicate relationship strength (e.g., smaller time gaps betweeninteractions, improved wellness evaluations following interactions, moresignificant interactions (e.g., mediagrams sent frequently)), andweighted less for interactions that indicated relationship weakness(e.g., longer time gaps between interactions, decreased or flat wellnessevaluations following interactions, less significant interactions). Forexample, allocated points can be weighted based on time intervalsbetween interactions (2008) and allocated points can be weighted basedon correlations between wellness ratings and interactions (2010). Otherweighting schemes are also possible.

The trend of weighted point allocations over time can be determined byevaluating a time series of weighted points for the relationship (2012).If the relationship is trending toward improvement—meaning that theweighted point allocations generally increase over time—then additionalpositive trend points can be awarded to the relationship (2014).

Weighted points can be aggregated (2016) and used to determine arelationship rating (2018). For example, the aggregate weighted pointscan be evaluated over the time period within which they occur todetermine one or more normalized statistics for the relationship (e.g.,average weighted points per time unit (e.g., day, week, month), medianpoint value, standard deviation of point values). The relationshiprating can be output and used to infer the state of the relationship(2020).

FIGS. 21A-B are flowcharts of example techniques 2100 and 2150 forcreating and using private group walls on a social platform.

Referring to FIG. 21A, the technique 2100 is one in which user-initiatedgroup creation takes place. A user selects an option to create a groupwall (2102) and the user (now the group creator) designates users to bea part of the group (2104). A prompt can be determined for the groupbased on the users in the group (2106). For example, a prompt toinitialize social interactions on the group wall can be determined basedon interests for users in the group. The prompt can be provided to one,some, or all of the users in the group (2108). In response to injectingthe prompt into the group wall, users can transmit responses and otherinteractions that are inserted into the group wall as well (2110).

The group wall can include self-policing features by which group memberscan flag inappropriate content and/or inappropriate members for thegroup. Flagged content can be provided to the creator and/or other usersin the group for review and possible deletion (2112). Similarly, flaggedusers can be provided to the creator and/or other users in the group forreview and possible removal from the group. Suggestions for additionalor new users to be added to the group can also be provided to thecreator and/or other users in the group for approval (2114). The steps2160-2114 can repeat for a threshold period of time, after which thegroup wall can automatically end (2116).

Referring to FIG. 21B, the technique 2150 is one in which automatic(non-user-initiated) group creation takes place. To automaticallyinitiate a group, user profiles and relationship profiles can beanalyzed to identify users to automatically include in the group (2152).For example, users with common interests between each other and one ormore preexisting connections to one or more other people in the pool ofcandidates for the group can be added to the group (each member of thegroup does not need a preexisting connection with each other member ofthe group). For example, a concierge-created group wall can be createdwith users who are fans of a sports team that recently won a big game orchampionship. The group can be automatically created and the members ofthe group can be notified (2154).

The concierge organizing the automatic group can seed the automaticallycreated group wall with a starting prompt (and subsequent follow-onprompts) (2156). Users can interact with each other on the group wall inresponse to the prompt (2158). One or more users of the group can bedesignated to moderate the group wall (2160). In some implementations,the group wall does not allow invitation of random or connectedadditional contacts. After a pre-set expiration time (e.g., 24 hours, 2days, 7 days), the group can end automatically (2162).

FIG. 22 is a block diagram of example computing devices 2200, 2250 thatmay be used to implement the systems and methods described in thisdocument, as either a client or as a server or plurality of servers.Computing device 2200 is intended to represent various forms of digitalcomputers, such as laptops, desktops, workstations, personal digitalassistants, servers, blade servers, mainframes, and other appropriatecomputers. Computing device 2200 is further intended to represent anyother typically non-mobile devices, such as televisions or otherelectronic devices with one or more processers embedded therein orattached thereto. Computing device 2250 is intended to represent variousforms of mobile devices, such as personal digital assistants, cellulartelephones, smartphones, and other computing devices. The componentsshown here, their connections and relationships, and their functions,are meant to be examples only, and are not meant to limitimplementations of the inventions described and/or claimed in thisdocument.

Computing device 2200 includes a processor 2202, memory 2204, a storagedevice 2206, a high-speed controller 2208 connecting to memory 2204 andhigh-speed expansion ports 2210, and a low-speed controller 2212connecting to low-speed bus 2214 and storage device 2206. Each of thecomponents 2202, 2204, 2206, 2208, 2210, and 2212, are interconnectedusing various busses, and may be mounted on a common motherboard or inother manners as appropriate. The processor 2202 can processinstructions for execution within the computing device 2200, includinginstructions stored in the memory 2204 or on the storage device 2206 todisplay graphical information for a GUI on an external input/outputdevice, such as display 2216 coupled to high-speed controller 2208. Inother implementations, multiple processors and/or multiple buses may beused, as appropriate, along with multiple memories and types of memory.Also, multiple computing devices 2200 may be connected, with each deviceproviding portions of the necessary operations (e.g., as a server bank,a group of blade servers, or a multi-processor system).

The memory 2204 stores information within the computing device 2200. Inone implementation, the memory 2204 is a computer-readable medium. Inone implementation, the memory 2204 is a volatile memory unit or units.In another implementation, the memory 2204 is a non-volatile memory unitor units.

The storage device 2206 is capable of providing mass storage for thecomputing device 2200. In one implementation, the storage device 2206 isa computer-readable medium. In various different implementations, thestorage device 2206 may be a floppy disk device, a hard disk device, anoptical disk device, or a tape device, a flash memory or other similarsolid state memory device, or an array of devices, including devices ina storage area network or other configurations. In one implementation, acomputer program product is tangibly embodied in an information carrier.The computer program product contains instructions that, when executed,perform one or more methods, such as those described above. Theinformation carrier is a computer- or machine-readable medium, such asthe memory 2204, the storage device 2206, or memory on processor 2202.

The high-speed controller 2208 manages bandwidth-intensive operationsfor the computing device 2200, while the low-speed controller 2212manages lower bandwidth-intensive operations. Such allocation of dutiesis an example only. In one implementation, the high-speed controller2208 is coupled to memory 2204, display 2216 (e.g., through a graphicsprocessor or accelerator), and to high-speed expansion ports 2210, whichmay accept various expansion cards (not shown). In the implementation,low-speed controller 2212 is coupled to storage device 2206 andlow-speed bus 2214. The low-speed bus 2214 (e.g., a low-speed expansionport), which may include various communication ports (e.g., USB,Bluetooth®, Ethernet, wireless Ethernet), may be coupled to one or moreinput/output devices, such as a keyboard, a pointing device, a scanner,or a networking device such as a switch or router, e.g., through anetwork adapter.

The computing device 2200 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 2220, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 2224. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 2222. Alternatively, components from computing device 2200 maybe combined with other components in a mobile device (not shown), suchas computing device 2250. Each of such devices may contain one or moreof computing devices 2200, 2250, and an entire system may be made up ofmultiple computing devices 2200, 2250 communicating with each other.

Computing device 2250 includes a processor 2252, memory 2264, aninput/output device such as a display 2254, a communication interface2266, and a transceiver 2268, among other components. The computingdevice 2250 may also be provided with a storage device, such as amicro-drive or other device, to provide additional storage. Each of thecomponents 2250, 2252, 2264, 2254, 2266, and 2268, are interconnectedusing various buses, and several of the components may be mounted on acommon motherboard or in other manners as appropriate.

The processor 2252 can process instructions for execution within thecomputing device 2250, including instructions stored in the memory 2264.The processor may also include separate analog and digital processors.The processor may provide, for example, for coordination of the othercomponents of the computing device 2250, such as control of userinterfaces, applications run by computing device 2250, and wirelesscommunication by computing device 2250.

Processor 2252 may communicate with a user through control interface2258 and display interface 2256 coupled to a display 2254. The display2254 may be, for example, a TFT LCD display or an OLED display, or otherappropriate display technology. The display interface 2256 may compriseappropriate circuitry for driving the display 2254 to present graphicaland other information to a user. The control interface 2258 may receivecommands from a user and convert them for submission to the processor2252. In addition, an external interface 2262 may be provided incommunication with processor 2252, so as to enable near areacommunication of computing device 2250 with other devices. Externalinterface 2262 may provide, for example, for wired communication (e.g.,via a docking procedure) or for wireless communication (e.g., viaBluetooth® or other such technologies).

The memory 2264 stores information within the computing device 2250. Inone implementation, the memory 2264 is a computer-readable medium. Inone implementation, the memory 2264 is a volatile memory unit or units.In another implementation, the memory 2264 is a non-volatile memory unitor units. Expansion memory 2274 may also be provided and connected tocomputing device 2250 through expansion interface 2272, which mayinclude, for example, a subscriber identification module (SIM) cardinterface. Such expansion memory 2274 may provide extra storage spacefor computing device 2250, or may also store applications or otherinformation for computing device 2250. Specifically, expansion memory2274 may include instructions to carry out or supplement the processesdescribed above, and may include secure information also. Thus, forexample, expansion memory 2274 may be provide as a security module forcomputing device 2250, and may be programmed with instructions thatpermit secure use of computing device 2250. In addition, secureapplications may be provided via the SIM cards, along with additionalinformation, such as placing identifying information on the SIM card ina non-hackable manner.

The memory may include for example, flash memory and/or MRAM memory, asdiscussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 2264, expansionmemory 2274, or memory on processor 2252.

Computing device 2250 may communicate wirelessly through communicationinterface 2266, which may include digital signal processing circuitrywhere necessary. Communication interface 2266 may provide forcommunications under various modes or protocols, such as GSM voicecalls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, orGPRS, among others. Such communication may occur, for example, throughtransceiver 2268 (e.g., a radio-frequency transceiver). In addition,short-range communication may occur, such as using a Bluetooth®, WiFi,or other such transceiver (not shown). In addition, GPS receiver module2270 may provide additional wireless data to computing device 2250,which may be used as appropriate by applications running on computingdevice 2250.

Computing device 2250 may also communicate audibly using audio codec2260, which may receive spoken information from a user and convert it tousable digital information. Audio codec 2260 may likewise generateaudible sound for a user, such as through a speaker, e.g., in a handsetof computing device 2250. Such sound may include sound from voicetelephone calls, may include recorded sound (e.g., voice messages, musicfiles, etc.) and may also include sound generated by applicationsoperating on computing device 2250.

The computing device 2250 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 2280. It may also be implemented as part of asmartphone 2282, personal digital assistant, or other mobile device.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. Other programming paradigms can be used, e.g., functionalprogramming, logical programming, or other programming. As used herein,the terms “machine-readable medium” “computer-readable medium” refers toany computer program product, apparatus and/or device (e.g., magneticdiscs, optical disks, memory, Programmable Logic Devices (PLDs)) used toprovide machine instructions and/or data to a programmable processor,including a machine-readable medium that receives machine instructionsas a machine-readable signal. The term “machine-readable signal” refersto any signal used to provide machine instructions and/or data to aprogrammable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular implementations of particularinventions. Certain features that are described in this specification inthe context of separate implementations can also be implemented incombination in a single implementation. Conversely, various featuresthat are described in the context of a single implementation can also beimplemented in multiple implementations separately or in any suitablesub-combination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

Thus, particular implementations of the subject matter have beendescribed. Other implementations are within the scope of the followingclaims. In some cases, the actions recited in the claims can beperformed in a different order and still achieve desirable results. Inaddition, the processes depicted in the accompanying figures do notnecessarily require the particular order shown, or sequential order, toachieve desirable results. In certain implementations, multitasking andparallel processing may be advantageous.

What is claimed is:
 1. A system for generating digital media files, thesystem comprising: a digital media file repository storing a pluralityof preselected digital video files that are excerpts of longer digitalvideo files, the plurality of preselected digital video files beingencoded in a common digital video codec and being stored with metadatathat identifies times within the plurality of preselected digital videofiles at which emotionally impactful moments occur; a frontend system incommunication with client computing devices, the frontend systemreceiving digital media file content generation requests from the clientcomputing devices that include parameters identifying particularpreselected digital video files to be combined with personal digitalmedia files to generate personalized digital video files, the personaldigital media files including personal digital video files, personaldigital audio files, personal text, and personal digital image filesthat are uploaded to the frontend system by the client computingdevices, the personal digital video files being encoded across aplurality of digital video codecs, a backend system to generate thepersonalized digital video files using the particular preselecteddigital video files and the personal digital media files, the backendsystem being programmed to: convert the personal digital video filesfrom the plurality of digital video codecs to the common video codec,retrieve personalization digital media templates that designate (i)particular types of media content to be combined with the particularpreselected digital video files and (ii) particular times withinparticular preselected digital video files at which the particular typesof media content are to be combined with the particular preselecteddigital video files, the particular times being relative to the timeswithin the plurality of preselected digital video files at which theemotionally impactful moments occur, assemble digital media content forthe personalized digital video files using particular preselecteddigital video files, the digital media templates, and the personaldigital media files, the personal digital media files being (i)positioned at the particular times relative to the times at which theemotionally impactful moments occur in the particular preselecteddigital video files, (ii) visually combined with video tracks of theparticular preselected digital video files so that digital images andvideos from the personal digital media files replace the video tracks atthe particular times, and (iii) audibly combined with audio tracks forthe particular preselected digital video files so that audio from thepersonal digital media files are automatically mixed with the audiotracks at the particular times, wherein the video tracks and the audiotracks for the particular preselected digital video files at the timesat which the emotionally impactful moments occur remain unmodified inthe assembled digital media content, encode the assembled digital mediacontent using the common video codec to generate the personalizeddigital video files, and store the personalized digital video files; anda digital media distribution system to transmit the personalized digitalvideo files to the client computing devices.
 2. The system of claim 1,wherein: the longer digital video files are full-length music videoscontaining full-length songs, and the plurality of preselected digitalvideo files are excerpts of the full-length music videos that includethe emotionally impactful moments.
 3. The system of claim 1, wherein:the longer digital video files are full-length movies, and the pluralityof preselected digital video files are excerpts of the full-lengthmovies that include the emotionally impactful moments.
 4. The system ofclaim 1, wherein: the personalized digital video files comprisemediagrams that include a personalized message centered around theemotionally impactful moments in the particular preselected digitalvideo files, and the mediagrams are configured to be digitally sent fromone client computing device to another client computing device.
 5. Thesystem of claim 1, wherein: the personal digital media files havevariable lengths of time, and assembling the digital media contentfurther includes adding one or more portions of digital filler contentso as (i) to fit the personal digital media files with variable lengthsof time at the particular times according to the digital media templatesand (ii) ensure that the video tracks and the audio tracks for theparticular preselected digital video files at the times at which theemotionally impactful moments occur remain unmodified in the assembleddigital media content.
 6. The system of claim 5, wherein the one or moreportions of digital filler content are loops of digital content derivedfrom the particular preselected digital video files.
 7. The system ofclaim 5, wherein the one or more portions of digital filler content arepreselected loops of digital content.
 8. A method for generating digitalmedia files, the method comprising: receiving digital media file contentgeneration requests from client computing devices that includeparameters identifying particular preselected digital video files to becombined with personal digital media files to generate personalizeddigital video files, the personal digital media files including personaldigital video files, personal digital audio files, personal text, andpersonal digital image files, the personal digital video files beingencoded across a plurality of digital video codecs, the preselecteddigital video files being excerpts of longer digital video files, thepreselected digital video files being encoded in the common digitalvideo codec and being stored with metadata that identifies times withinthe preselected digital video files at which emotionally impactfulmoments occur; converting the personal digital video files from theplurality of digital video codecs to a common video codec; retrievingpersonalization digital media templates that designate (i) particulartypes of media content to be combined with the particular preselecteddigital video files and (ii) particular times within particularpreselected digital video files at which the particular types of mediacontent are to be combined with the particular preselected digital videofiles, the particular times being relative to the times within theplurality of preselected digital video files at which the emotionallyimpactful moments occur; assembling digital media content for thepersonalized digital video files using particular preselected digitalvideo files, the digital media templates, and the personal digital mediafiles, the personal digital media files being (i) positioned at theparticular times relative to the times at which the emotionallyimpactful moments occur in the particular preselected digital videofiles, (ii) visually combined with video tracks of the particularpreselected digital video files so that digital images and videos fromthe personal digital media files replace the video tracks at theparticular times, and (iii) audibly combined with audio tracks for theparticular preselected digital video files so that audio from thepersonal digital media files are automatically mixed with the audiotracks at the particular times, wherein the video tracks and the audiotracks for the particular preselected digital video files at the timesat which the emotionally impactful moments occur remain unmodified inthe assembled digital media content; encoding the assembled digitalmedia content using the common video codec to generate the personalizeddigital video files; storing the personalized digital video files; andtransmitting the personalized digital video files to the clientcomputing devices.
 9. The method of claim 8, wherein: the longer digitalvideo files are full-length music videos containing full-length songs,and the plurality of preselected digital video files are excerpts of thefull-length music videos that include the emotionally impactful moments.10. The method of claim 8, wherein: the longer digital video files arefull-length movies, and the plurality of preselected digital video filesare excerpts of the full-length movies that include the emotionallyimpactful moments.
 11. The method of claim 8, wherein: the personalizeddigital video files comprise mediagrams that include a personalizedmessage centered around the emotionally impactful moments in theparticular preselected digital video files, and the mediagrams areconfigured to be digitally sent from one client computing device toanother client computing device.
 12. The method of claim 8, wherein: thepersonal digital media files have variable lengths of time, andassembling the digital media content further includes adding one or moreportions of digital filler content so as (i) to fit the personal digitalmedia files with variable lengths of time at the particular timesaccording to the digital media templates and (ii) ensure that the videotracks and the audio tracks for the particular preselected digital videofiles at the times at which the emotionally impactful moments occurremain unmodified in the assembled digital media content.
 13. The methodof claim 12, wherein the one or more portions of digital filler contentare loops of digital content derived from the particular preselecteddigital video files.
 14. The method of claim 12, wherein the one or moreportions of digital filler content are preselected loops of digitalcontent.
 15. A computer program product encoded on a non-transitorystorage medium, the product comprising non-transitory, computer readableinstructions for causing one or more processors to perform operationscomprising: receiving digital media file content generation requestsfrom client computing devices that include parameters identifyingparticular preselected digital video files to be combined with personaldigital media files to generate personalized digital video files, thepersonal digital media files including personal digital video files,personal digital audio files, personal text, and personal digital imagefiles, the personal digital video files being encoded across a pluralityof digital video codecs, the preselected digital video files beingexcerpts of longer digital video files, the preselected digital videofiles being encoded in the common digital video codec and being storedwith metadata that identifies times within the preselected digital videofiles at which emotionally impactful moments occur; converting thepersonal digital video files from the plurality of digital video codecsto a common video codec; retrieving personalization digital mediatemplates that designate (i) particular types of media content to becombined with the particular preselected digital video files and (ii)particular times within particular preselected digital video files atwhich the particular types of media content are to be combined with theparticular preselected digital video files, the particular times beingrelative to the times within the plurality of preselected digital videofiles at which the emotionally impactful moments occur; assemblingdigital media content for the personalized digital video files usingparticular preselected digital video files, the digital media templates,and the personal digital media files, the personal digital media filesbeing (i) positioned at the particular times relative to the times atwhich the emotionally impactful moments occur in the particularpreselected digital video files, (ii) visually combined with videotracks of the particular preselected digital video files so that digitalimages and videos from the personal digital media files replace thevideo tracks at the particular times, and (iii) audibly combined withaudio tracks for the particular preselected digital video files so thataudio from the personal digital media files are automatically mixed withthe audio tracks at the particular times, wherein the video tracks andthe audio tracks for the particular preselected digital video files atthe times at which the emotionally impactful moments occur remainunmodified in the assembled digital media content; encoding theassembled digital media content using the common video codec to generatethe personalized digital video files; storing the personalized digitalvideo files; and transmitting the personalized digital video files tothe client computing devices.
 16. The computer program product of claim15, wherein: the longer digital video files are full-length music videoscontaining full-length songs, and the plurality of preselected digitalvideo files are excerpts of the full-length music videos that includethe emotionally impactful moments.
 17. The computer program product ofclaim 15, wherein: the longer digital video files are full-lengthmovies, and the plurality of preselected digital video files areexcerpts of the full-length movies that include the emotionallyimpactful moments.
 18. The computer program product of claim 15,wherein: the personalized digital video files comprise mediagrams thatinclude a personalized message centered around the emotionally impactfulmoments in the particular preselected digital video files, and themediagrams are configured to be digitally sent from one client computingdevice to another client computing device.
 19. The computer programproduct of claim 15, wherein: the personal digital media files havevariable lengths of time, and assembling the digital media contentfurther includes adding one or more portions of digital filler contentso as (i) to fit the personal digital media files with variable lengthsof time at the particular times according to the digital media templatesand (ii) ensure that the video tracks and the audio tracks for theparticular preselected digital video files at the times at which theemotionally impactful moments occur remain unmodified in the assembleddigital media content.
 20. The computer program product of claim 19,wherein the one or more portions of digital filler content are loops ofdigital content derived from the particular preselected digital videofiles.